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Tutorial Announcement 



DIMACS Tutorial 

Limits of Approximation Algorithms: PCPs and Unique Games 

DIMACS Center, CoRE Building, Rutgers University, July 20 - 21, 2009 

Organizers: 

* Prahladh Harsha, University of Texas, Austin 

* Moses Charikar, Princeton University 

This tutorial is jointly sponsored by the DIMACS Special Focus on Hardness of Approxima- 
tion, the DIMACS Special Focus on Algorithmic Foundations of the Internet, and the Center for 
Computational Intractability with support from the National Security Agency and the National 
Science Foundation. 

The theory of NP-completeness is one of the cornerstones of complexity theory in theoretical 
computer science. Approximation algorithms offer an important strategy for attacking compu- 
tationally intractable problems, and approximation algorithms with performance guarantees have 
been designed for a host of important problems such as balanced cut, network design, Euclidean 
TSP, facility location, and machine scheduling. Many simple and broadly-applicable approximation 
techniques have emerged for some provably hard problems, while in other cases, inapproximability 
results demonstrate that achieving a suitably good approximate solution is no easier than finding 
an optimal one. The celebrated PCP theorem established that several fundamental optimization 
problems are not only hard to solve exactly but also hard to approximate. This work shows that 
a broad class of problems is very unlikely to have constant factor approximations, and in effect, 
establishes a threshold for such problems such that approximation beyond this threshold would 
imply P= NP. More recently, the unique games conjecture of Khot has emerged as a powerful 
hypothesis that has served as the basis for a variety of optimal inapproximability results. 

This tutorial targets graduate students and others who are new to the field. It will aim to give 
participants a general overview of approximability, introduce them to important results in inapprox- 
imability, such as the PCP theorem and the unique games conjecture, and illustrate connections 
with mathematical programming techniques. 

List of speakers: Matthew Andrews (Alcatel-Lucent Bell Laboratories), Sanjeev Arora (Prince- 
ton University), Moses Charikar (Princeton University), Prahladh Harsha (University of Texas, 
Austin), Subhash Khot (New York University), Dana Moshkovitz (Princeton University) and Lisa 
Zhang (Alcatel-Lucent Bell Laboratories) 
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Lecture 1 



An Introduction to Approximation Algorithms 
Sanjeev Arora 



Scribe: Darakhshan J. Mir 
20 July, 2009 



In this lecture, we will introduce the notion of approximation algorithms and see examples of 
approximation algorithms for a variety of NP-hard optimization problems. 

1.1 Introduction 

Let Q be an optimization problem 1 . An optimal solution for an instance of this optimization 
problem is a feasible solution that achieves the best value for the objective function. Let OPT {I) 
denote the value of the objective function for an optimal solution to an instance /. 

Definition 1.1.1 (Approximation ratio). An algorithm for Q has an approximation ratio a if for 
instances I, the algorithm produces a solution of cost < a ■ OPT (I) (a > 1), if Q is a minimization 
problem and of cost > a ■ OPT{I) if Q is a maximization problem. 

We are interested in polynomial-time approximation algorithms for NP-hard problems. How 
does a polynomial-time approximation algorithm know what the cost of the optimal solution is, 
which is NP-hard to compute? How does one guarantee that the output of the algorithm is within 
a of the optimal solution when it is NP-hard to compute the optimal solution. In various examples 
below, we see techniques of handling this dilemma. 

1 Formally, a (maximization) optimization problem is specified by two domains X,y, a feasibility function feas : 
X x 3^ — > {0, 1} and an evaluation function value : X x y — > R. An input instance to the problem is an element 
x 6 X. For each such x, the optimization problem is as follows: 

OPT(x) = max {value (x, y) \ y £ y ,ieas(x,y) — 1} . 

OPT(x) is also called the optimal value. 
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1.1.1 Examples 

1. 2-approximation for metric Travelling Salesman Problem (metric-TSP): Consider 
a complete graph G formed by n points in a metric space. Let dij be the distance between 
point i and j. The metric TSP problem is to find a minimum cost cycle that visits every 
point exactly once. 

The following observation relating the cost of the minimum spanning tree (MST) to the 
optimal TSP will be crucial in bounding the approximation ratio. 

Observation 1.1.2. The cost of the Minimum spanning Tree (MST) is at most the optimal 
cost of TSP. 

Algorithm A: 

(a) Find the MST 

(b) Double each edge 

(c) Do an "Eulerian transversal" and output its cost 
Observe that TSP < cost(A) < 2 • MST < 2 • TSP. 

2. A 1.5-approximation to metric-TSP: The approximation ratio can be improved to 1.5 
by modifying the above using an idea due to Christofides [Chr76]. Instead of doubling each 
edge of the MST as in the above algorithm, a minimum cost matching is added among all 
odd degree nodes. Observe that cost of matching < ^TSP. So, 

Cost(Appx-algo) < MST + ]^TSP < 1.5 • TSP 

It is to be noted that since 1976, there has been no further improvement on this approximation 
ratio. 

The above examples are examples of approximation algorithms that attain a constant approx- 
imation ratio. In the next section, we will see how to get arbitrarily close to the optimal solution 
when designing an approximation algorithm, ie., approximation ratios arbitrarily close to 1. 

1.2 Polynomial-time Approximation Scheme (PTAS) 

A PTAS is a family of polynomial-time algorithms, such that for every e > 0, there is an algorithm 
in this family that is an (1 + e) approximation to the NP-hard problem Q, if it is a minimization 
problem and an (1 — e)-approximation if Q is a maximization problem. 

The above definition allows the running time to arbitrarily depend on e but for each e it should 

-L 1 

be polynomial in the input size e.g. or n 2e . 
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1.2.1 Type-1 PTAS 



Various type of number problems typically have type-1 PTAS. The usual strategy is to try to round 
down the numbers involved , so the choice of numbers is small and then use Dynamic Programming. 
The classic example of such an approach is the Knapsack problem. 

Knapsack problem Given a set of n items, of sizes si, S2 ■ ■ ■ s n such that S{ < 1 V«, and profits 
ci, C2, . . . , Cn, associated with these items, and a knapsack of capacity 1, find a subset / of items 
whose total size is bounded by 1 such that the total profit is maximized. 

The knapsack problem is NP-hard in general, however if the profits fall in a small-sized set, 
then there exists an efficient polynomial time algorithm. 

Observation 1.2.1. // the values c±,C2, ■ ■ ■ c n are in [1, . . . ,w], then the problem can be solved in 
poly (n,w) -time using dynamic programming. 

This naturally leads to the following approximation algorithm for knapsack. 

(1 + e)-Approximation Algorithm 

1. Let c = maxj Cj. 

2. Round down each a to the nearest multiple of ^ . Let this quantity be rj-(^) , i.e., rj = |_ c i/^J- 

3. With these new quantities (rj) as profits of items, use the standard Dynamic Programming 
algorithm, to find the most profitable set I' . 

The number of r^s is at most n/e. Thus, the running time of this algorithm is at most 
poly(n, n/e) = poly(n, 1/e). We now show that the above algorithm obtains a (1 — e)-approximation 
ratio 

Claim 1.2.2. Ylier Ci ^ an 0- ~ e )~ approximation to OPT. 

Proof. Let O be the optimal set. For each item, rounding down of Cj causes a loss in profit of at 
most — . Hence the total loss due to rounding down is at most n times — . In other words, 



The first inequality follows from the definition of rj, the second from the fact that I' is an optimal 
solution with costs rj's, the third from the above observation and the last from the fact that 




Hence, f £ ie0 n > OPT - ce. Now, 




a > — A n > — \ n > OPT - ec> {\-e)OPT 





OPT > c. 



□ 
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1.2.2 Type-2 PTAS 



In these kinds of problems we define a set of "simple" solutions and find the minimum cost simple 
solution in polynomial time. Next, we show that an arbitrary solution may be modified to a simple 
solution without greatly affecting the cost. 

Euclidean TSP A Euclidean TSP is a TSP instance where the points are in M. 2 and the distances 
are the corresponding Euclidean distances. 

A trivial solution can be found in n\. Dynamic Programming finds a solution in n 2 2 n . 

We now give a high-level description of a n 1//e -time algorithm that achieves a (l+e)-approximation 
ratio. Consider the smallest square that contains all n points. Use quad-tree partitioning to re- 
cursively partition each square into four subsquares until unit squares are obtained. We consider 
the number of times the tour path crosses a cell in the quad-tree. We construct the "simple solu- 
tion" to the problem by restricting the tour to cross each dividing line < - times. We can then 
discretize the lines at these crossing points. Each square has < — number of crossing points. A 
tour may use each of these crossing points either 0, 1 or 2 times. So for the entire quadtree there 
are < 3^ = exp(^) number of possibilities. For details see Arora's 2003 survey [Aro03]. 

In the next section, we will see examples of approximation algorithms which use linear pro- 
gramming and semi-definite programming. 

1.3 Approximation Algorithms for MAXCUT 

The MAX-CUT problem is as follows: Given a graph G(V, E) with |V| = n, find m&xscv\E(S, S)\. 
The notation E(S, S) refers to the set of all edges (i, j) such that vertex i £ S and vertex j £ S. 

1.3.1 Integer Program Version 

Define variable Xi, such that Xi = 0, if vertex i € S and X{ = 1, if i G S. We have the following 
integer program: 




(ij)<EE 

subject to 



eij < min{xi + Xj, 2 — (x{ + Xj)}, V(i, j) G E 
xi,..., x n £ {0,1} 
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1.3.2 Linear Program Relaxation and Randomized Rounding 



This can be converted to a Linear Program as follows: 



Maximize e ij 

(ij)dE 

subject to 

eij < mm{xi + Xj,2 - (xi + Xj)}, £ E 

xi,...,x n £ [0,1] 

Every solution to the Integer Program is also a solution to the Linear Program. So the objective 
function will only rise. If OPTlp is the optimal solution to the LP, then: 

OPTlp > MAX-CUT 



Randomized Rounding 

We now round the LP-solution to obtain an integral solution as follows: form a set S by putting 
i in S with probability x; L . The expected number of edges in such a cut, E[|^(S I , S)\] can be then 
calculated as follows: 

E[|£(5,5)|] = Yl is in the cut] 

= ^ Xi{l - Xj) + Xj(l - Xi) 

The above calculates only an expected value of the cut, however if we repeat the above algorithm 
several times, it can be seen by Markov's inequality that we can get we can get very close to this 
value. We now show that this expected value is at least half the LP-optimal, which in turns means 
that it is at least half the MAX-CUT 

Claim 1.3.1. 

E[\E(S, S)\} = " x j) + " x i) > ^OPT LP > ^MAX-CUT 

(ij)eE 

Proof. We have 

OPT L p = ^ = ^ min{(xj + Xj), 2 - (xi + Xj)} 

It can easily be checked that for any Xi,Xj £ [0, 1], we have 

Xi(l - xj) + Xj(l - Xi) > ^ • min{(xj + Xj),2 - (xj + Xj)}. 

Thus, a term by term comparison of the LHS of the inequality with OPTlp reveals that E[|i£(£, S)\] > 
\OPT LP > |MAX-CUT. □ 

We thus, have a 1 /2-approximation algorithm for MAX-CUT using randomized rounding of the 
LP-relaxation of the problem. Actually, it is to be noted that the LP-relaxation is pretty stupid, the 
optimal to the LP is the trivial solution x% = 1/2 for all i, which in turn leads to OPTlp = \E\. But 
we do mention this example as it naturally leads to the following more powerful SDP relaxation. 
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1.3.3 Semi Definite Programming (SDP) Based Method 

We will now sketch a 0.8 78- approximation to MAX-CUT due to Goemans and Williamson [GW95]. 
The main idea is to relax the integer problem defined above using vector valued variables. THE 
SDP relaxation is as follows: 

(1 — (Vi,Vj)) 

Maximize > - x ' 31 ' 

^ 2 

subject to {vi,Vi) = 1, Vi 

Denote the optimal to the above SDP by OPTsdp- We first observe that the SDP is in fact a 
relaxation of the integral problem. Let vq be any vector of unit length, i.e., (vq,vq) = 1. Consider 
the optimal cut S that achieves MAX-CUT. Now define, 

/ v Q if i € S w . 
Vi = < _ . , ,Vz. 

{—vo if i f b 

Consider the quantity — ^ ,v ^\ This is if the vectors Vi and Vj lie on the same side, and equals 
1 if they lie on opposite sides. Thus, OPT SD p > MAX-CUT. 

How do we round the SDP solution to obtain an integral solution. The novel rounding due 
to Goemans and Williamson is as follows: The SDP solution produces n vectors v±, . . . ,v n . Now 
pick a random hyperplane passing through the origin of the sphere and partition vectors according 
to which side tof the hyperplane they lie. Let (S, S) be the cut obtained by the above rounding 
scheme. It is easy to see that 

E[\E(S,S)\] = Yl Pr[(i,i)€cut] 

= ^2 P T [vi,Vj he on opposite sides of the hyperplane] 

Let 9ij be the angle between vectors V{ and Vj . Then the probability that they are cut is proportional 
to Oij, in fact exactly Oij/n. Thus, 



E[\E(S,S)\]= E 

(ij)<EE 



TT 



Let us know express OPTsdp in terms of the Since 0^ = cos 1 ((vi,Vj)), we have 

op TsDP = y: (1 ^f sM 

By a "miracle of nature" (Mathematica?) Goemans and Williamson observed that 

- > (0.878 . . .) x 1 ~ C °^ , ye G [0, tt] 

7T 2 

Hence, 

Thus, we have a 0.878-approximation algorithm for MAX-CUT. 
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Lecture 2 

The PCP Theorem: An Introduction 



Dana Moshkovitz 



Scribe: Alexander S. Kulikov 
20 Jul, 2009 



Complementing the first introduction lecture on approximation algorithms, this lecture will be an 
introduction to the limits of approximation algorithms. This will in turn naturally lead to the PCP 
Theorem, a ground-breaking discovery from the early 90's. 

2.1 Optimization Problems and Gap Problems 

The topic of this lecture is the hardness of approximation. But to talk about hardness of approx- 
imation, we first need to talk about optimization problems. Recall the definition of optimization 
problems from the earlier lecture. Let us begin by giving an example of a canonical optimization 
problem. 

Definition 2.1.1 (MAX-3SAT). The maximum 3-satisfiability problem (MAX-3SAT) is: Given a 
3-CNF formula 4> (each clause contains exactly three literals) with m clauses, what is the maximum 
fraction of the clauses that can be satisfied simultaneously by any assignment to the variables? 

We first prove the following important claim. 

Claim 2.1.2. There exists an assignment that satisfies at least 7/8 fraction of clauses. 

Proof. The proof is a classical example of the probabilistic method. Take a random assignment 
(each variable of a given formula is assigned either or 1 randomly and independently). Let Yi be 
a random variable indicating whether the i-th clause is satisfied. For any 1 < i < m (where m is 
the number of clauses), 

1 7 
EYi = • - + 1 • - , 
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as exactly one of eight possible assignments of Boolean constants to the variables of the i-ih. clause 
falsifies this clause. Here we use the fact that each clause contains exactly three literals. 

Now, let Y be a random variable equal to the number of satisfied clauses: Y = YliL t ^i- Then, 
by linearity of expectation, 

mm „ 

EY = E^ Yi = EY i = — ■ 

1=1 i=l 

Since a random assignment satisfies a fraction 7/8 of all clauses, there must exist an assignment 
satisfying at least as many clauses. □ 

The natural question to ask is if we can do better? Can we find an assignment that satisfies 
more clauses. Let us phrase this question more formally. For this, we first recall the definition of 
approximation algorithms from the previous lecture. 

Definition 2.1.3. An algorithm C for a maximization optimization problem is called a- approximation 
(where < a < 1), if for every input x, the algorithm C outputs a value which is at least a times 
the optimal value, i.e., 

a ■ OPT(x) < C(x) < OPT(x) . 

Claim 2.1.2 implies immediately that there exists an efficient (i.e., polynomial time) 7/8- 
approximation algorithm for MAX-3SAT. The natural question is whether there exists an approxi- 
mation algorithm that attains a better approximation ratio. The answer is that such an algorithm 
is not known. The question that we are going to consider in this lecture is whether we can prove 
that such an algorithm does not exist. Of course, if we want to prove this, we have to assume that 
P^NP, because otherwise there is an efficient 1-approximation algorithm. 

There is some technical barrier here. We are talking about optimization problems, i.e., problems 
where our goal is to compute something. It is however much more convenient to consider decision 
problems (or languages), where we have only two possible answers: yes or no. So, we are going 
to transform an optimization problem to a decision problem. Namely, we show that hardness of 
a certain decision problem implies hardness of approximation of the corresponding optimization 
problem. 

Definition 2.1.4. For a maximization problem I and A < B € M + , the corresponding [A,B]-gap 
problem is the following promise decision problem 1 : 

YES = {x\OPT(x) > B} 
NO = {x|OPT(x) < A} 

We now relate the hardness of the maximization problem to the hardness of the gap problem. 

Theorem 2.1.5. If the [A,B]-gap version of a maximization problem is NP-hard, then it is NP- 
hard to approximate the maximization problem to within a factor A/B. 

X A promise problem IT is specified by a pair (YES, NO) where YES, NO G {0, 1}* and YES and NO are disjoint 
sets. Note there is no requirement that YES U NO = {0, 1}*. This is the only difference between promise problems 
and languages. 
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Proof. Assume, for the sake of contradiction, that there is a polynomial time ^/^-approximation 
algorithm C for a maximization problem under consideration. We are going to show that this 
algorithm can be used in order to solve the gap problem in polynomial time. 

The algorithm for the gap problem is: for a given input x, if C(x) > A, return "yes", otherwise 
return "no". 

Indeed, if x is a yes-instance for the gap problem, i.e., OPT(x) > B, then 

C(x) > A/B • OPT(x) > A/B ■ B = A 
and we answer "yes" correctly. If, on the other hand, OPT(x) < A, then 

C(x) < OPT(x) < A 

and we give the correct "no" answer. □ 

Thus, to show hardness of approximation to within a particular factor, it suffices to show 
hardness of the corresponding gap problem. Hence from now onwards, we focus on gap problems. 

2.2 Probabilistic Checking of Proofs 

We will now see a surprising alternate description of the hardness of gap problems. The alternate 
description is in terms of probabilistically checkable proofs, called PCPs for short. 

2.2.1 Checking of Proofs 

Let us first recall the classical notion of proof checking. NP is the class of languages that have a 
deterministic polynomial-time verifier. For every input x in the language, there exists a proof that 
convinces the verifier that x is in the language. For every input x not in the language, there is no 
proof that convinces the verifier that x is in the language. 

For example, when the language is 3SAT, the input is a 3CNF formula (p. A proof for the 
satisfiability of ip is an assignment to the variables that satisfies (p. 

A verifier that checks such a proof may need to go over the entire proof before it can know 
whether ip is satisfiable: the assignment can satisfy all the clauses in ip, but the last one to be 
checked. 

2.2.2 Local Checking of Proofs 

Can we find some other proof for the satisfiability of <p that can be checked locally, by querying 
only a constant number of symbols from the proof? 

For this to be possible, we allow the queries to be chosen in a randomized manner (otherwise, 
effectively the proof is only of constant size, and a language that can be decided using a polynomial- 
time verifier with access to such a proof can be decided using a polynomial-time algorithm). The 
queries should be chosen using at most a logarithmic number of random bits. The logarithmic 
bound ensures that the verifier can be, in particular, transformed into a standard, deterministic 
polynomial time, verifier. The deterministic verifier would just perform all possible checks, instead 
of one chosen at random. Since the number of random bits is logarithmic, the total number of 
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possible checks is polynomial. The number of queries the deterministic verifier makes to the proof 
is polynomial as well. 

To summarize, we want a verifier that given the input and a proof, tosses a logarithmic number 
of random coins and uses them to make a constant number of queries to the proof. If the input 
is in the language, there should exist a proof that the verifier accepts with probability at least B. 
If the input is not in the language, for any proof, the verifier should accept with probability at 
most A. The error probability is the probability that the verifier does not decide correctly, i.e., 
1 — B + A. If B = 1, we say that the verifier has perfect completeness, i.e., it never errs on inputs 
in the language. 

2.2.3 The Connection to The Hardness of Gap Problems 

The A^P-hardness of approximation of 3SAT is in fact equivalent to the existence of local verifiers 
for NP: 

Hardness =>- Local Verifier 

For [A, l]-gap-MAX-3SAT, there is a verifier that makes only 3 queries to the proof, has perfect 
completeness, and errs with probability at most A\ 

On input formula ip, the proof is a satisfying assignment for <p. The verifier chooses a random 
clause of </>, reads the assignment to the three variables of the clause, and checks if the clause is 
satisfied. The verifier uses log m random bits, where m is the number of clauses. If p is satisfiable, 
the verifier accepts the proof with probability 1. If not, at most A fraction of all clauses of <f> can 
be satisfied simultaneously, so the verifier accepts with probability at most A. 

Moreover, the NP-hardness of [A, l]-gap-MAX-3SAT yields local verifiers for all NP languages! 
More precisely, 

Claim 2.2.1. // [A, l]-gap-MAX-3SAT is NP-hard, then every NP language L has a probabilis- 
tically checkable proof (PCP). That is, there is an efficient randomized verifier that uses only 
logarithmic number of coin tosses and queries 3 proof symbols, such that 

• if x £ L, then there exists a proof that is always accepted; 

• if x £ L, then for any proof the probability to err and accept is at most A. 

Note that the probability of error can be reduced from A to e by repeating the action of the 
verifier k = 0( j°^j^ ) times, thus making O(k) queries. 

Local Verifier =>• Hardness 

What about the other direction? Do local verifiers for NP imply the NP-hardness of gap-MAX- 
3SAT, which would in turn imply inapproximability of MAX-3SAT? 

For starters, let us assume that every language in NP has a verifier that makes three bit queries 
and whose acceptance predicate is the OR of the three variables (or their negations). Assume 
that for inputs in the language the verfier always accepts a vaild proof, while for inputs not in 
the language, for any proof, the verifier accepts with probability at most A. From the above 
correspondence between local verifiers and gap problems, we get that [A, l]-gap-MAX-3SAT is 
NP-hard. 
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What if the verifier instead reads a constant number of bits (not 3) and its acceptance predicate 
is some other Boolean function on these constant number of bits? We will now use the fact that 
every Boolean predicate /: {0, -»• {0, 1} can be written as a 3-CNF formula <f> with clauses 
(and some additional variables z) such that for any assignment x of Boolean values to variables of 
/, f(x) = 1 iff <j)(x,z) is satisfiable for some z. We have thus transformed the 0(l)-query verfier 
with an arbitrary Boolean acceptance predicate to a 3-query verifier with acceptance predicate an 
OR of the 3 variables (or their negations). We thus have. 

Claim 2.2.2. If every NP language L has a constant query verifier that uses only logarithmic 
number of coin tosses and queries Q proof symbols, such that 

• if x £ L, then there exists a proof that is always accepted; 

• if x L, then for any proof the probability to err and accept is at most A. 

then [A', l]-gap-MAX-3SAT is NP-hard for some A' < 1. 

Thus, we have shown that the problem of proving NP-hardness of gap-MAX-3SAT is equivalent 
to the problem of constructing constant query verifiers for NP. But do such verifiers exist? 

2.2.4 The PCP Theorem 

Following a long sequence of work, Arora and Safra and Arora, Lund, Motwani, Sudan and Szegedy 
in the early 90's constructed local verifiers for NP: 

Theorem 2.2.3 (PCP Theorem (. . . ,[AS98],[ALM+98])). Every NP language L has a probabilisti- 
cally checkable proof (PCP). More precisely, there is an efficient randomized verifier that uses only 
logarithmic number of coin tosses and queries 0(1) proof symbols, such that 

• if x £ L, then there exists a proof that is always accepted; 

• if x L, then for any proof the probability to accept it is at most 1/2. 

The proof in [AS98, ALM+98] is algebraic and uses properties of low-degree polynomials. There 
is a more recent alternate combinatorial proof for the theorem due to Dinur [Din07] . We will later 
in the workshop see some of the elements that go into the construction. 

The PCP Theorem shows that it is NP-hard to approximate MAX-3SAT to within some constant 
factor. The natural further question is: can we improve this constant to 7/8 (to match the trivial 
approximation algorithm from Claim 2.1.2)? A positive answer to this question would yield a tight 
7/8- hardness for approximation of MAX-3SAT. 

2.3 Projection Games 

In 1995, Bellare, Goldreich, and Sudan [BGS98] introduced a paradigm for proving inapproximabil- 
ity results. Following this paradigm, Hastad [HasOl] established tight hardness for approximating 
MAX-3SAT, as well as many other problems. 

The paradigm is based on the hardness of a particular gap problem, called Label- Cover. 
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Definition 2.3.1. An instance of a projection game (also called label-cover ) is specified by bipartite 
graph G = (A,B,E), two alphabets T*a, £b> an d projections 7r e : T,a — > (for every edge e £ E). 

Given assignments A : A — > Y,a and B : B — > Y*b, an edge e = (a,b) G E is said to be satisfied 
iff Tr e (A(a)) = B(b). The value of this game is 

max I Pr \e is satisfied] ) . 

In a label-cover problem; given a projection game instance, compute the value of the game. 




The PCP-theorem could be formulated as a theorem about Label-Cover. 

Theorem 2.3.2 (. . . , [AS98], [ALM + 98]). Label-Cover is NP-hard to approximate within some 
constant. 

Proof. The proof is by reduction to Label-Cover. Recall that we have a PCP verifier from the 
previous formulation. We construct a bipartite graph as follows. For each possible random string 
of the verifier, we have a vertex on the left-side. Since the verifier uses only a logarithmic number 
of bits, the number of such strings is polynomial. For each proof location, we have a vertex on the 
right-side. We add an edge between a left vertex (i.e., a random string) and a right vertex (i.e., a 
proof location), if the verifier queries this proof location on this random string. Thus, we defined 
a bipartite graph. We are now going to describe the labels and projections. 

The labels for the left-side vertices are accepting verifier views and for the right-side are proof 
symbols. A projection is just a consistency check. For example, in case of MAX-3SAT, we have 
satisfying assignments of a clause on the left-side and values of variables on the right-side. □ 

However, this theorem is not strong enough to get tight hardness of approximation results. For 
this reason, we call this the weak projection games theorem. What we actually need is a low error 
version of this theorem, which improves the hardness in Theorem 2.3.2 from "some constant" to 
"any arbitrarily small constant". 

Theorem 2.3.3 ((strong) Projection Games Theorem (aka Raz's verifier) [Raz98]). For every 
constant e > 0, there exists a k = k(e) such that is is NP-hard to decide if a given projection game 
with labels of size at most k (ie., |XU|, |£b| < k) has value 1 or at most e. 
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The PCP construction in the strong projection games theorem is commonly refered to as Raz's 
verifier as the theorem follows from Raz's parallel repetition theorem applied to the construction 
in Theorem 2.3.2 [Raz98]. Recently, Moshkovitz and Raz [MR08] gave an alternate proof of this 
theorem that allows the error e to be sub-constant. 

Why does the label size k depend on e in the above theorem? This is explained by the following 
claim, which implies that k must be at least 1/e. 

Claim 2.3.4. There is an efficient \/k- approximation algorithm for projection games on labels of 
size k (i.e., |Xa|> < k). 

We will later in this tutorial see how the projection games theorem implies tight hardness of 
approximation for 3SAT. 

We remark that for many other problems, like Vertex-Cover or Max-Cut, we do not know 
of tight hardness of approximation results based on the projection games theorem. To handle such 
problems, Khot formulated the unique games conjecture [Kho02]. This conjecture postulates the 
hardness of unique label cover, where the projections ir e on the edges are permutations. More on 
that - later in the tutorial. 
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Lecture 3 



Approximation Algorithms for Network Problems 
Matthew Andrews 

Scribe: Gwen Spencer 
20 July, 09 



Lectures 3 and 4 will be on network/flow problems, the known approximation algorithms and 
inapproximability results. In particular, this lecture will serve as an introduction to the different 
types of network/flow problems and a survey of the known results, while the follow-up lecture by 
Lisa Zhang will deal with some of the techniques that go into proving hardness of approximation 
of network/flow problems. 

3.1 Network Flow Problems 

Network/Flow problems are often motivated by industrial applications. We are given a communi- 
cation or transportation network and our goal is to move/route objects/information though these 
networks. 

The basic problem that we shall be considering is defined by a graph G = (V, E) and a set of 
(source, destination) pairs of nodes which we'll denote (s±,ti), (52^2)5 etc. We will sometimes call 
these pairs "demand pairs." There are many variants of the problem: 

• Only Connectivity is required. The question is one of feasibility: "Is it possible to select 
a subset of the edge set of G that connects every (sj, tj) pair?" 

• Capacities must be respected. Each edge has a capacity, and each (si,ti) pair has some 
amount of demand that must be routed from Sj to ij. Observe that this problem is infeasible 
if there exists a cut in the graph which has less capacity than the demand which must cross 
it. Imagine variations on this problem in which more capacity can be purchased on an edge 
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at some cost (that is, the capacities are not strict): the question becomes: "What is the 
minimum amount of capacity that must be purchased to feasibly route all demand pairs?" 

• What solutions are "good" depends on the objective function. Consider the differ- 
ence between the objective of trying to minimize the maximum congestion (where congestion 
is the total demand routed along an edge) and the objective of trying to minimize the total 
capacity purchased: it is not hard to find examples where a good solution with respect to 
the first objective is a bad solution with respect to the second objective and vice versa. The 
maximum congestion objective is often used to describe delay/quality of service. 

• Splittable vs. unsplittable flow. In the unsplittable flow case all demand routed between 
Si and ti must travel on a single (s«, ti) path. In the splittable flow case each demand can be 
split so that it is routed on a set of paths between Sj and ti . 

• Directed vs. Undirected. Is the graph directed or undirected? As a rule of thumb, 
problems in which the graph is directed are more difficult. 

Next we'll consider some specific problems and describe what positive and negative results exist 
for each of them: 

3.1.1 Minimum Cost Steiner Forest 

In this problem we are interested in simple connectivity. The input to the problem is a graph with 
edge costs and a set of (sj, ti) pairs. The goal is to connect each (sj, ti) pair via a set of edges which 
has the minimum possible total cost (the cost of a set of edges is just the sum of the costs of all 
edges in the set). 

Notice that any feasible solution to this problem is a set of trees. 

Both positive and negative results exist for this problem: 

• Positive: 2-approximation (Agrawal-Klein-Ravi [AKR95], Goemans- Williamson [GW95]) 

• Negative: APX-hard, there exists e such that no 1 + e approximation algorithm exists for 
the problem unless P=NP. 

3.1.2 Congestion Minimization (Fractional) 

The input to this problem is a graph with edge capacities and a set of (sj,ij) pairs. The goal is to 
connect all (sj, tj) pairs fractionally (that is, for all i, to route a total of one unit of demand from S{ 
to ti along some set of paths in the graph) in a way that minimizes the maximum congestion. The 
congestion on an edge is simply the total demand routed on that edge divided by the capacity of 
the edge. The maximum congestion is the maximum congestion taken over all edges in the graph. 

This problem can be solved exactly in polynomial time via a linear program. We write the 
linear program as follows: let u e denote the capacity of edge e, and have a decision variable x Pi i 
which is the amount of demand i that is routed on path p: 

min z 

T.iT,p:e&p X P,i< ZU e Ve. 

%p,% > Vp, i. 



15 



The first set of constraints says that for each demand pair i, one unit of demand must be routed 
from Si to tj. The second set of constraints says that for each edge e, the sum of all demand routed 
on e must be less than z times the capacity of e. Since the objective is to minimize z, the optimal 
LP solution finds the minimum multiplicative factor z required so that the capacity of each edge 
is at least 1/z times the total demand routed on that edge (that is, the optimal z is the minimum 
possible maximum congestion). 

Though this LP is not of polynomial size (the number of paths may be exponentially large) it 
can be solved in polynomial time, using an equivalent edge-based formulation whose variables y e ,i 
represent the amount of flow from demand i routed through edge e. Hence we can obtain an exact 
solution to the problem. 

3.1.3 Congestion Minimization (Integral) 

Now consider the Congestion Minimization problem when we require the routing be integral (all 
demand routed from Sj to t\ must be routed on a single path). We can no longer solve this problem 
using the linear program above. The following results are known: 

• Positive: A 0(logn/loglogn)-approximation algorithm where n is the number of vertices 
due to Raghavan-Thompson [RT87]. This algorithm is based on the technique of randomized 
rounding which we describe below. 

• Negative: Andrews-Zhang [AZ07]) show that there is no (log n) ^^approximation unless 
NP has efficient algorithms. More formally our result holds unless NPCZPTIME(nP°^ Zo 9( n )), 
where ZPTIME(nP°^ io s( n )) is the class of lan guages that can be recognized by random- 
ized algorithms that always give the correct answer and whose expected running time is 
n poi y iog(n) _ n io g k n for gome constant fc- The assumption that NP£ZPTIME(nP°^( n )) is 

not quite as strong an assumption as NP ^ P but is still widely believed to be true. 

Note that the gap between the positive and negative results here is large. We comment that for 
the directed version of the problem, a negative result has been proved that no f2(logre/loglogn)- 
approximation exists unless NP has efficient algorithms [CGKT07, AZ08]. 

We now describe the Raghavan-Thompson randomized rounding method for approximating the 
Integral Congestion Minimization Problem: 

• Notice that the LP for the fractional problem is a linear relaxation of the IP we would write 
for the integral case. Thus, the optimal solution to the fractional version is a lower bound on 
the optimal value of the integral version: OPTf rac < OPTi n t egra i- 

• Note that ^p x p,« = 1 f° r au *• 'Lreat the probability distribution: demand i is 
routed on path p with probability x p ^. By linearity of expectation, the expected congestion 
of the resulting ranodmly rounded solution on each edge is at most OPTf rac . 

In any given rounding though, some edges will have more than their expected congestion. It is 
possible to show that for any fixed edge e, with large probability (> 1 — ^) the congestion on 
edge e is 0(log nj log log n)OPTf rac = 0(log nj log log n)OPTi ntegra i. By a union bound this 
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implies that with probability at least \ the maximum congestion of the randomly rounded 
solution on any edge is O '(log n/ log log n)OPTi nte g ra i- 

For the directed case this gives the best achievable approximation. Whether something better 
exists for the undirected case is an open question. 

3.1-4 Edge Disjoint Paths 

The input is a graph with edge capacities and a set of (si,ti) pairs. The goal is to connect every 
(si,ti) pair integrally using disjoint paths (that is, to find a set of paths, one connecting each pair, 
such that the paths for two distinct pairs share no edges). The goal is to connect the maximum 
possible number of pairs. 

The following results are known: 

• Positive: A 0(m 1 / 2 )-approximation where m is the number of edges, due to Kleinberg [Kle96] . 

• Negative: (undirected) No (logn) 1 / 2 "^ -approximation exists unless NP has efficient algo- 
rithms (Andrews- Zhang [AZ06]). 

• Negative: (directed) No 0(m 1 / 2 ~ £ )-approximation exists unless P=NP (Guruswami-Khanna- 
Rajaraman-Shepherd-Yannakakis [GKR + 03] ) . 

We'll look at Kleinberg's m 1 / 2 approximation algorithm for this problem (m is the number of 
edges). Consider a greedy algorithm as follows: 

1. Find the shortest path that connects two terminals. 

2. Remove all the edges on that path from the graph. 

3. Repeat until we cannot connect any more terminals. 

Analysis. At all times we let G' be the subgraph of G that contains the remaining edges (i.e. 
the edges that have not been removed in Step 2). There are two cases in our analysis: either the 
shortest path p linking two terminals in the remaining graph G' has length at most m 1 / 2 , or not: 

• Suppose the shortest path p in G' has length < m 1 / 2 . Each edge in p intersects at most one 
path from the optimal solution (since the paths in the optimal solution are disjoint), so p 
intersects at most m 1 / 2 paths from the optimal solution. Thus, when the algorithm removes 
p, at most m 1 / 2 paths are removed from the optimal solution. 

Thus, the algorithm produces at least one path for every m 1 / 2 paths in the optimal solution. 

• Suppose the shortest path p in Q has length strictly greater than m 1 / 2 . Since the paths in 
the optimal solution are disjoint, and all must have length at least as long as p, the optimal 
solution has at most m/((m 1 / 2 )) = m 1 ' 2 paths in G . Thus, to get a m 1 ' 2 approximation for 
G' the algorithm need only produce one path (so the algorithm can just use p). 

We mention that for this problem we can't hope to do better with a linear programming relax- 
ation method because the gap between the optimal IP solution and the optimal LP solution can 
be m 1 / 2 . 
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3.1.5 Minimum Cost Network Design 



The input is a graph, a set of (si,ti) pairs and a cost function /(c) for placing capacity on an edge. 
The goal is to route one unit of demand between each pair in a way that requires the minimum 
cost expenditure for capacity. 

Commonly considered cost functions include: 

1. Linear: shortest paths are optimal. 

2. Constant: this results in the Steiner forest problem. 

3. Subadditive: economies of scale and buy-at-bulk problems. This is a nice way of modelling 
how aggregating demand onto a core network is beneficial and arises in many industrial 
network design problems. For subadditive cost functions the following results are known: 

• Positive: a 0(logre)-approximation (Awerbuch-Azar [AA97], Bartal [Bar98], Fakcharoenphol- 
Rao-Talwar [FRT04]). 

• Negative: No (logn) 1//4_e approximation exists unless NP has efficient algorithms (An- 
drews [And04]). 

Summary 

Approximation ratios vary widely for different types of network flow problems: 

• Constant approximation: Steiner forest. 

• 0(log n)-approximation: Congestion minimization, Buy-at-Bulk network design. 

• m 1 / 2 -approximation: Edge Disjoint paths. 



Questions 

Q: Are these algorithms actually what is used in practice? 

Answer: Not exactly. Take the case of the randomized rounding that we covered: in practice 
this technique may not give the best congestion due to the Birthday Paradox. It is quite likely that 
there is some edge that gets higher congestion than the average by a logarithmic factor. Hence, 
practical algorithms typically apply heuristics to try and reduce the congestion. One technique 
that often works well is to sort the demands based on the distance between the terminals (from 
closest to farthest). We then go through the demands in order and try to greedily reroute them. 

We remark that industrial networks often cost a huge amount of money and so tweaking a 
solution a little to save even a single percent can generate meaningful cost savings. In addition, a 
lot of these real applications are huge: cutting-edge computing power together with CPLEX are 
not even close to being able to solve these problems exactly. Approximation really is necessary. 
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Lecture 4 



Hardness of the Edge- Disjoint Paths Problem 
Lisa Zhang 



Scribe: David Pritchard 
20 July, 2009 



4-1 Overview 

The edge- disjoint paths problem (EDP) is the combinatorial optimization problem with inputs 

• a (directed or undirected) graph G with n nodes and m edges 

• a list of k pairs of (not necessarily distinct) nodes of G, denoted (sj, U)f =1 
and whose output is 

• a subset X of {1, . . . , k} representing a choice of paths to route 

• Si-ti paths {Pi}i £ x so that the Pi are pairwise edge-disjoint 
and 

• the objective is to maximize \X\. 

In these notes, the main results are: a simple proof that for any e > it is NP-hard to approximate 
the directed edge-disjoint paths problem to ratio m 1 / 2-6 (Section 4.3); and a more complex proof 
that for any e > 0, if we could approximate the undirected edge-disjoint paths problem to ratio 
log 1 / 3_e n, then there would be randomized quasi-polynomial time algorithms for NP (Section 4.4). 
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4.2 Literature 



For directed EDP, there is a simple y'm-approximation algorithm due to Kleinberg [Kle96] (see also 
Erlebach's survey [Erl06]), which nearly matches the m 1 / 2_e -hardness result we will present (which 
is due to Guruswami et al. [GKR + 03]). A 0(ra 2 / 3 log 2 / 3 n) approximation is also known [VV04]. 

For undirected EDP, Kleinberg's simple algorithm [Kle96] still gives a -^/m-approximation, 
but an improved -^/n-approximation was recently obtained by Chekuri et al. [CKS06]. The main 
technical ingredient in the proof we will present is the high girth argument, which was used first in 
2004 by Andrews [And04] and subsequently in a variety of papers [ACG+07, AZ06, AZ07, AZ08, 
AZ09, CK06, GT06b], some of which have closed the approximability and inapproximability gaps of 
various problems up to constant factors. Many of these papers deal with congestion minimization, 
where all demands must be routed and the objective is to minimize the maximum load on any edge. 
Focusing on undirected graphs, the papers most closely related to what we will show are: 

• [And04], which introduced the high girth argument and gave a polylog-hardness for buy-at- 
bulk undirected network design. For this problem, all demands must be routed, and the cost 
minimized. The types of "buy-at-bulk" edges used in the hardness construction were fixed- 
cost edges (which once bought, can be used to any capacity) and linear-cost edges (where you 
pay proportional to the capacity used) . The paper reduced from a type of 2-prover interactive 
system, similar to PCPs. 

• [AZ06], which gave the log 1 / 3-6 n-hardness proof we will describe in these notes. The paper 
reduced from Trevisan's inapproximability results [TreOl] on the bounded degree independent 
set problem. In turn, those resuls rely on advanced PCP technology [ST00]. 

• [ACG + 07] — a paper which was the culmination of merging several lines of work — which 
resulted in an improved log 1//2_e n-hardness proof for undirected EDP. This paper uses the 
hardness of constraint satisfaction problems, while the preliminary versions use the Raz ver- 
ifier (parallel repetition) and directly-PCP based methods. This is so far the best inap- 
proximability result known for undirected EDP, although it is very far from the best known 
approximation ratio of y/n [CKS06] . 

In more detail, the table below summarizes some results in the literature (lower bounds assume 
NP <f_ ZPTIME(n polylogn ), and some constant factors are omitted). Stars (*) denote results in which 
the high girth method is used. 

Problem 

Undirected EDP 
Directed EDP 

Undirected Congestion Minimization 
Directed Congestion Minimization 
Undirected Uniform Buy-at-Bulk 
Undirected Nonuniform Buy-at-Bulk 
Undirected EDP with Congestion c 
(with some restrictions on c) 
Directed EDP with Congestion c 
(with some restrictions on c) 

If the number k of terminal pairs is fixed, the undirected EDP problem is exactly solvable 
in polynomial-time, using results from the theory of graph minors [RS95]. (As we will see in 



Upper bound 

m 1/2 [Kle96], n 1/2 [CKS06] 

m 1/2 [Kle96], n 2/3 log 2/3 n [VV04] 

log m/ log log m [RT87] 

log m/ log log m [RT87] 

log m [AA97, FRT04] 

log 5 m [CHKS06] 



Lower Bound 

★ log 1/3 - f m [AZ06], *log 1/2 - f m [ACG+07] 



m 



1/2-e 



[GKR+03] 



logm [CGKT07] 



.1— e , 



rn 



[AZ08], 



*log 1/4_E m [And04] 
*log 1/2_E m [And04] 



log log m 



,1/c 



[AR06, BS00, KS01] 
[AR06, BS00, KS01] 



rlog (1 - £)/(c+1) m [ACG+07] 
m n(1/c) [CGKT07] 
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Figure 4.1: Left: the graph G in an instance of Dir-2EDP. Right: The construction of H for k = 5. 
Each small yellow object is a copy of G, and each other line is a directed edge. This illustration is 
adapted from Erlebach [Erl06]. 

Theorem 4.3.2, the directed case behaves differently.) 

4.3 Hardness of Directed EDP 

In this section we prove the following theorem. 

Theorem 4.3.1 ([GKR + 03]). For any e > it is NP-hard to approximate the directed edge-disjoint 
paths problem (Dm-EDP) to within ratio m l l 2 ~ e . 

Although it is very common for inapproximability proofs in the literature to reduce one approx- 
imation problem to another, this proof has the cute property that it reduces an exact problem to 
an approximation problem. Phrased differently, the complete proof does not rely on any PCP-like 
technology. Specifically, our starting point is the following theorem. 

Theorem 4.3.2 ([FHW80]). The following decision problem (Dir-2EDP) is NP-hard: given a 
directed graph G and four designated vertices s,s',t,t' in the graph, determine whether there are 
two edge-disjoint directed paths, one from s to t, and another from s' to t' . 

(Note, this immediately shows that it is hard to NP-hard to approximate (Dir-EDP) to a 
factor better than 2.) 

The key to proving Theorem 4.3.1 is a construction which maps (G, s, s' ,t,t') to an instance 
{H, (s l ,t i % 1 ) of Dir-EDP where k is a parameter we will tune later. The construction is illustrated 
in Figure 4.1. The two important properties of this construction are the following: 

(a) If G admits edge-disjoint s-t and s'-t' paths (say P and P'), then H has a solution of value 
k (i.e. all pairs Sj-£j for 1 < i < k can be simultaneously linked by edge-disjoint paths). To 
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see this, we utilize the copies of P and P' within each copy of G; then it's easy to see there 
are mutually disjoint paths Si-U paths (the path leaving s, goes up through i—1 copies of P, 
then right through k — i copies of P', to ti). 

(b) If G does not admit edge-disjoint s-t and s'-tf paths, then there is no solution for H with 
value greater than 1. To see this, suppose for the sake of contradiction that there is an Sj-ij 
path Pi and a Sj-tj path Pj in H such that Pi, Pj are edge-disjoint. Without loss of generality 
i < j. Then a topological argument shows that there must be some copy of H such that P{ 
uses the copies of s' and t! and Pj uses the copies of s and t. This contradicts our assumption 
about the Dm-2EDP instance. 

Facts (a) and (b) show that any algorithm that has approximation ratio better than on the 
Dm-EDP instance H also solves the DIR-2EDP instance. 

Without loss of generality we assume G is (weakly) connected, then the encoding size of the 
Dir-2EDP instance is proportional to and the encoding size of the Dir-EDP instance is 

\E(H)\ = 0(k 2 \E(G)\). In order to conclude that it is NP-hard to approximate the Dir-EDP 
instance to a factor better than k, we need \E(H)\ to be polynomial in Thus we may take 

a 

k = \E(G)\ a for any constant a. Going back to the analysis, we get k = \E(H)\ 2a + 1 ; hence by 
taking a — > oo, we get the desired result (that it is NP-hard to approximate Dir-EDP to a factor 
k = m l l 2 - e ). 

4-4 Hardness of Undirected EDP 

In this section we sketch the proof of the following theorem. 

Theorem 4.4.1 ([AZ06]). For any e > 0, if we can approximate the directed edge-disjoint paths 
problem (UNDIR-EDP) to within ratio 0(log 1 ^ 3_e m), then every problem in NP has a probabilistic 
always-correct algorithm with expected running time exp(polylog(n)) ; i.e. NP C ZPTIME(exp(polylog( 

We fully describe the construction of the proof and give intuition for the analysis, but skip some 
of the detailed parts and precise setting of parameters. The construction creates a simple graph (i.e. 
one with no parallel edges) so the theorem also holds with log m replaced by log n since these are the 
same up to a factor of 2. The proof shows more precisely that NP C coRPTIME(exp(polylog(n))), 
i.e. it gives a quasi-polynomial size, randomized reduction with one-sided error that is right at least 
(say) 2/3 of the time; then standard arguments 1 allow us to move to ZPTIME. Here is the proof 
overview. 

• The starting point is the inapproximability of the independent set problem (IS) in bounded- 
degree graphs: find a set of mutually non-adjacent vertices (an independent set) with as large 
cardinality as possible. We denote the degree upper bound by A. 

1 We insert Q into standard complexity class names to denote quasi-polynomial time. Suppose NP C coRQP. Then 
there is a /(n)-time algorithm for SAT with / quasi-polynomial. This also implies NQP C coRQP since every NQP 
language with quasi-polynomial running time g(n) is equivalent (by the Cook-Levin construction) to satisfiability 
of a formula of size g(n), and it can be decided in time f(g(n)) which is quasi-polynomial. The definition of RQP 
immediately implies RQP C NQP hence RQP C coRQP. Taking complements we deduce RQP = coRQP and it is 
easy to show that RQP n coRQP = ZPQP, hence NP C coRQP = ZPQP. Alternatively, see Lemma 5.8 in [EH03] for 
a more efficient construction. 
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• As usual, our goal is to find a transformation from IS instances to Undir-EDP instances 
which preserves the "NP-hard-to-distinguish gap" in the objective function. 

• We will create a new graph G in the following way. Roughly we "define a path" Pj for each 
vertex of the IS instance so that Pi n P, 7^ iff V{,Vj are adjacent. (It is easy to see this 
is possible, with the length of P, proportional to the degree of V{.) Then we define G to be 
the union of all Pj. (See Figure 4.2.) 

• To get some intuition for the rest of the proof, define the terminals Sj, U of the Undir-EDP 
instance to be the endpoints of Pj. Then it is almost true that the Undir-EDP instance is 
isomorphic (in terms of feasible solutions) to the IS instance. The significant problem is that 
G necessarily also contains Sj-tj paths other than Pj, which may be used for routing. (Such 
paths are obtained by using a combination of edges taken from different P/'s; see Figure 4.2.) 

• To get around this problem, we transform G into a different graph H defined by two parame- 
ters x, c. Each intersection of two paths Pj, Pj is replaced by c consecutive intersections; and 
we replace each Pj with x images {Pi, a }%=i- The construction of H has a lot of independent 
randomness, two consequences of which are that (i) when Vi,Vj are adjacent, we can lower- 
bound the probability that Pj jQ n Pj j( g 7^ and (ii) H has few short cycles. We call each 
Pi, a a canonical path; for each canonical path its endpoints define a terminal pair for the new 
Undir-EDP instance. 

• The optimum of the Undir-EDP solution is at least x times the optimum of the IS instance. 
To get our hardness-of-approximation result, we also need that when the Undir-EDP opti- 
mum is "large", so is the IS optimum. This is done via a map from Undir-EDP solutions 
R on H to IS solutions S on G. The map is parameterized by a number a < x. We (1) 
throw away all non-canonical paths in R and (2) take Vi in S iff at least a out of the x paths 
{Pi,«}a=i are routed by Y. The final analysis uses the fact that the canonical paths have 
length 0(cA) while most non-canonical paths are long; the latter depends on the fact that H 
has few short cycles. 

4-4-1 Hardness of Bounded- Degree Independent Set 

Trevisan [TreOl] showed that for any constant A, it is NP-hard to approximate the independent set 
problem within ratio A/2 v/log A on graphs with degrees bounded by A. For our purposes, we need 
a version of this that works for super-constant A. By extending the framework in Trevisan [TreOl], 
Andrews and Zhang [AZ06] proved the following: 

Theorem 4.4.2. Consider the family of graphs with upper bound A = log b n on degree, where n 
is the number of nodes and b is a constant. If there is a (log 6_e re)- approximation algorithm for IS 
on these graphs for any e > 0, then NP C coRPTIME(re°( loglogn )). 

44.2 The Graphs G and H 

First we give a formal description of the graph G we sketched earlier. Let Go denote the IS 
instance, without loss of generality Go is connected. Each edge e = ViVj of Go yields two vertices 
u \hj} ; w bJ} g V(G) and each vertex vi of Go yields two vertices Sj, £»; these are all the vertices 



23 



S3 S 2 




Figure 4.2: An illustration of how the independent set instance, Go (left), yields a Undir-EDP 
instance on graph G (right). The colour of m in Go corresponds to the colour of canonical path 
Pi in G. Note that in this example, even though {t>i,t>2} is not an independent set, there are 
edge-disjoint Sx-t% and S2-t2 paths (Pi, and a non-canonical S2-t2 path). 

of G. Let the neighbours of V{ in Go in any order be v p , v q , . . . , v r , then we define the path 
Pi := (sj, v,{ l ' p } , iy{ l ' p } , u{ l ' q } , w{ l ' q } , . . . , vS- i,r ^ , w^ l ' r ^ , U). More precisely, for each adjacent pair of 
vertices in this list we define an edge of G; this constitutes all of the edges of G. Every edge of 
the form uW/iyw'} appears in both Pj and P,, while every other edge of G appears in exactly one 
Pi. The number of vertices and edges of G is 0{\E(Gq)\) and the number k of terminal pairs is 
\V{G Q )\. 

There are two additional ideas needed to define H , one whose effect is to randomly replace Pi 
by x images {Pj jQ }^ =1 and another whose effect is to increase the probability that two paths Pi <a , 
Pj : p intersect when ViVj S E(Gq). 

Consider the following probabilistic operation f x on graphs: replace every vertex v by x "copies" 
{ v a}a=i, and replace every edge v v' with a random bipartite matching of {v a }^ =1 to {v' a }^ =1 , where 
these matchings are chosen independently for all input edges vv' . Thus f x multiplies the total 
number of vertices and edges by x. Note that if vv' is an edge of some graph K and 1 < a, f3 < x 
then the probability that u a vp G fx(K) is exactly 1/x; we will later use this fact, as well as the 
independence of the different random matchings, to show that f x (K) behaves like a random graph 
in terms of short cycles. We define the image of terminal pairs under f x as follows. Define Pj jQ 
as the unique path in f x (G) obtained by starting at Sj iQ (which denotes (si) a ) and following the 
images of edges of Pi. Pi )a does not necessarily end at U^ a , rather it ends at tj >( g =: t' ia for some 
uniformly random /3. The terminal pairs of f x (G) are all pairs (si, a , t[ ). We call the Pi )Ct canonical 
paths. 

At this point it is straightforward to compute the following: if v^j E E{Gq) and a, (3 are 
fixed, the probability that the paths Pj iQ ., Pj p intersect in f x (G) is exactly 1/x. More generally, 
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Figure 4.3: Illustrating, for c = 3, the operation used to transform G (left) into G' (right). 

for subsets A, B of {1, . . . , x}, the probability Pr[{Pj )a } a6j 4 U {P/^j^eB are mutually edge-disjoint 
in f x (G)] can be expressed as some function 6(x, \A\, \B\) 2 . We would like to decrease this (i.e. 
increase the probability some Pi iOC intersects some Pj,p), and to do so, we consider a graph G' 
obtained similarly to G except, for ViVj G E(Gq), we force Pi and Pj to intersect c times. We give 
an informal but precise definition since the formal definition is lengthy. To construct G 1 from G, 
we perform the following for all edges V{Vj G E{Gq): replace the intersection edge u^'^w^ 1 '^ with 
the gadget pictured in Figure 4.3, and simultaneously redefine Pj, Pj to follow the indicated paths. 
Not only will this cause the new Pi and Pj to intersect c times, but the images of these intersections 
under f x will be independent in the sense that, for all subsets A, B of {1, . . . , x}, the probability 
Pi[{Pi : a} a& A U {Pj,/3}/3eB are mutually edge-disjoint in f x (G')] decreases to <5 c (x, \A\, \B\). 

Finally, H is defined to be f x (G'), with canonical paths Pi a and terminal pairs defined as for 
SAG). 

4-4-3 Small Cycles 

The graph H = f x (G') we defined is not quite a "random graph" in the usual (Erdds-Renyi) 
sense, but it has enough randomness that it has a typical property of random graphs, namely 
that the number of small cycles can be bounded. This is done using the first moment method 
(Markov's inequality), analogous to the 1963 Erdos-Sachs theorem (e.g. that in the Erdds-Renyi 
model G(n, d/n) the number of cycles of length g is at most d 9 in expectation). 

If C is any simple cycle in G', then it is not hard to see that the expected number of simple 

Explicitly, 5{x, \A\, \B\) = (x- \A\)\(x - \B\)\/{x - \A\ - \B\)\xl. 
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cycles in H that are "images" of G is 1. This is a good bound but it is not quite sufficient for 
our purposes, since cycles may exist in H whose inverse image in G is not simple. To be precise 
about getting a bound, for each edge vv' of G , we say that it has x 2 corresponding potential edges 
{v a v'g | 1 < a, j3 < x] in H = f x (G). (In any realization of H, exactly x of these edges are actually 
present.) Then it is not hard to see that we get the following: conditioned on the existence of 
any k < x potential edges in H, the probability that any other potential edge is in H is at most 
l/(x — k). Then the first moment method allows us to show: 

if g < x/2, E[# cycles in H of length < g] < O(n cA) 9+1 (4.4.1) 

where we define no = |V(Go)|- Note that this bound is independent of x; roughly speaking this is 
because the factor of x in |V(iJ)| cancels with the factor 1/x in the probability of H containing 
any given potential edge. Eventually, we will set g to be poly- logarithmic in no, and the right-hand 
side of Equation (4.4.1) will be quasi-polynomial in uq. 

It is not hard to argue that H has maximum degree 3, so each vertex has 0{2 9 ) vertices within 
distance g; combining this fact with Equation (4.4.1) gives us the following form of the "high-girth 
argument" that we use in the final proof: 

Pr[0(nocA) s+1 vertices in H have distance < g to a cycle of length < g] > 9/10. (4.4.2) 



4-4-4 Analysis Sketch 

As mentioned earlier, our reduction uses the following map Ri-> S from Undir-EDP routings on 
H to independent sets on Go, parameterized by a number a: put V{ into S if at least a out of the x 
paths Pi >a for i are routed by R. To be exact, S is only an independent set with some probability, 
which we would like to make large. By applying simple bounds to 5, for any A, B C {1, . . . , x} with 
|^4|, \B\ > a and for i,j adjacent in Go, we have that 

Pr[{Pi, Q } aG A U {Pj^jpeB mutually edge-disjoint in f x (G)} < exp(-ca 2 /x). (4.4.3) 

For any two fixed adjacent vertices Vi,Vj in Go, by a union bound, the probability that any 
subsets A,B with > a exist, such that A and B fail the event in (4.4.3) is at most 

(a) (a) ex P( — co- 2 / x )- Therefore using another union bound, 

Pr[5 independent] > 1 - |£(G )| Q Q exp(-ca 2 /^). (4.4.4) 

The setting of parameters in the proof is then chosen so that Pr[S independent] is at least 9/10. 
This, along with Theorem 4.4.2 and Equation (4.4.2), are the three sources of error in the proof. 
At a high level the analysis breaks the paths in R into four types, 

(a) a canonical path Pi )Qt so that #{/S|Pj a G R} > a. 

(b) a canonical path Pj )Q so that #{(3\Pi r £ P} < a 

(c) a non-canonical Sj )Q ,-i^ a path where Si jQ , has distance < g to a cycle of length < g 

(d) any other non-canonical Sj )Q ,-^ a path 

and applies the following analysis (recall n = \E(Gq)\): 
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• The number of paths of type (a) is at most \S\x. 

• The number of paths of type (b) is at most (no — \S\)a. 

• The number of paths of type (c) is at most 0(uqcA) 9+1 by (4.4.2). 

• To upper bound the number of paths of type (d), let P' denote one such path. The union of 
P' and Pi ia contains a simple cycle, but the length of that cycle is at least g. The length of 
Pi >a is fixed at O(cA) and hence the length of P' is at least g — O(cA). Since the type-(d) 
paths are disjoint, there are at most \E(H)\/(g — 0(cA)) of them. 

This gives a lower bound on IS"! in terms of \R\. The proof is then completed by setting the 
parameters carefully. In detail, using the fact that the greedy algorithm for independent set on Go 
always gives a solution of value at least no/ (A + 1), we can show that |5| is an independent set 
with size at least a constant times \R\/x provided that g>A 2 c, x>Aa, c>^ In | + x n 2 n ° , x>A 2 c ■ 
0{uqAc) 9 hold, where we have omitted some constant factors. (These conditions come from the 
relative contributions of the different types of paths, as well as the error bounds.) The ratio of 
inapproximability for Undir-EDP is then roughly A as a function of the input size m = \E(H)\ = 
0(uqcAx), and it is not hard to show that A is roughly log 1//3 m at maximum. (In [AZ06], a precise 
setting of parameters is given.) 
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Proof of the PCP Theorem (Part I) 
Prahladh Harsha 



Scribe: Ashkan Aazami 
20 July, 2009 



In this lecture and the follow-up lecture tomorow, we will see a sketch of the proof of the PCP 
theorem. Recall the statement of the PCP theorem from Dana Moshkovitz's lecture earlier today. 
Dana had mentioned both a weak form (the original PCP Theorem) and a strong form (Raz's 
verifier or hardness of projective games). We need the strong form as it is the starting point of 
most tight inapproximability results. "Standard proofs" of the strong form proceed as follows: 
first prove the PCP Theorem [AS98, ALM+98] either using the original proof or the new proof of 
Dinur [Din07] and then apply Raz's parallel repetition [Raz98] theorem to it to obtain the strong 
form. However, since the work of Moshkovitz and Raz [MR08], we can alternatively obtain the 
strong form directly using the proof techniques in the orginal proof of the PCP Theorem along 
with the composition technique of Dinur and Harsha [DH09]. We will follow the latter approach in 
this tutorial. 



5.1 Probabilistically Checkable Proofs (PCPs) 

We first introduce the probabilistically checkable proof (PCP) and some variants of it. 

Our goal is to construct a PCP for some NP-complete problem. We will work with the NP- 
complete problem Circuit-SAT. Let C be an instance of the CiRCUiT-SAT problem. A PCP 
consists of a verifier V that is provided with a proof ir of acceptance of the input instance C. 
The goal of the verifier is to check if the given proof is "valid". Given the input C, the verifier 
V generates a random string R and based on the input instance C and the random bits of R it 
generates a list Q of queries from the proof ir. Next, the verifier V queries the proof ir at the 
locations of Q and based on the content of the proof in these locations the verifier either accepts 
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the input C as an acceptable instance or rejects it. The content of tt at the locations Q is called the 
local view of tt and it is denoted by ttq. We denote the local predicate that the verifier checks by <p; 
the verifier accepts if ^>(ttq) = 1 and it rejects otherwise. The verifier has the following properties: 

Completeness: If C is satisfiable then there is a proof tt such that the verifier always accepts with 
probability 1; i.e., 

3tt : Prob [<p(n Q ) = 1] = 1. 

Soundness: If C is not satisfiable then for every proof tt the verifier accepts with probability at 
most 5 (say 5 = |) ; 

Vvr : Prob [p{ttq) = !]<-• 



PCP 




V(C,R) = (Q,<p) 
7r|q: local view 
<p: local predicate 



V accepts if <p[tt\q) = 1 



2-provers projective PCP 



A : u A -> s 



fl : U B -> Si 




y accepts if ^^(^(m)) = 



Figure 5.1: PCP and 2-queries projective PCP 
The original PCP Theorem now can be stated formally as follows. 



Theorem 5.1.1 (PCP Theorem [AS98, ALM+98]). Circuit-SAT has a PCP (with the above 
completeness and soundness properties) that uses \R\ = O(logn) random bits and queries \Q\ = 
O(l) locations of the proof where n is the size of the input circuit. 

Note that the length of the proof tt is polynomial in n, the size of the input instance C, since 
the length of the random string is O(logn) so the verifier can make 2°( logra ) number of queries. 



5.1.1 Strong Form of the PCP Theorem and Robust PCPs 

Now we introduce a strong form of the PCP theorem, this is also called the 2-prover projection 
game theorem. In this type of PCPs, there are two non-communicating provers A : U a and 
B : Ub — > and a verifier V. Given the input instance C, the verifier first generates a random 
string R of length logarithmic in the input size and then using the random string, it determines two 
locations u and v and generates a projection function ir UyV : J] a — > £#• The verifier then queries 
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the two provers Ua and Ub on locations u and v respectively and accepts the provers' answers if 
they are consistent with the projection function ie., ir u ,v(A(u)) = B(v). 
We can now state the strong from of the PCP theorem as follows. 

Theorem 5.1.2 (Strong form of PCP (aka Raz's verifier, hardness of projection games) [Raz98]). 
For any constant 5 > 0, there exist alphabets E^,£s such that the Circuit-SAT has a 2-prover 
projection game with a verifier V such that 

Completeness: If C is satsifiable (C £ CiRCUiT-SATj then there exist two provers A : Ua — > 
T,a, B : Ub — > such that 

Prob[7r Ui „(A(u)) =B(v)\ = 1 

Soundness: If C is not satisiable (C ^ CiRCUlT-SATj then for all pairs of provers A : Ua — > 
T,a, B : Ub — > £b, we have 

Pvob[TT U)V (A(u)) = B(v)\ < 5. 

Now we introduce the notion of robust PCP. These PCPs have a stronger soundness property. 
In the ordinary PCPs, the soundness property says that if the input instance C is not an acceptable 
input, then the local predicate that the verifier checks is not satisfied with high probability. In the 
robust PCPs the local view is far from any satisfying assignment with high probability. 

First for some notation. Given two codewords x and y, the agreement between x and y is 
defined as agr(x,y) = Prob \x{ = yj\. For a given set S of code-words, we define the agreement of 

i 

S and x by agr(x, S) = max^gs agr(j;, y). Let us denote the set of all satisfiable assignments to the 
local predicate ip by SAT(Q). 

The robust PCPs have the same completeness property as in the ordinary PCPs, but they have 
a stronger soundness property. More precisely, the following soundness property of regular PCPs 
is replaced by the stronger "robust soundness" property. 

Soundness: 

C i Circuit-SAT =>- Prob [ir Q e SAT{tp)\ < 5 

Robust Soundness: 

C £ Circuit-SAT =>■ l[agr(7r Q , SAT(<p))] < 5 
We call PCPs with the robust soundness property, robust PCPs. 

5.1.2 Equivalence of Robust PCPs and 2-Provers Projection PCPs 

Note that robust PCPs are just regular PCPs with a stronger soundness requirement. We now 
show that robust PCPs are equivalent to 2-provers projection PCPs. Given a robust PCP with 
the verifier V and the prover tt, we construct a 2-prover projective verifier V' and two provers 
A, B as follows. The prover B is the same prover as it. For each possible random string R and 
the corresponding queries Q of the verifier V, the prover A has the local view ttq at the location 
indexed by R; i.e., the prover A has all possible local views of the prover tt. The verifier V of the 
2-prover projection PCP is as follows. 
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1. Generate a random string R and compute a set Q of queries as in the verifier V. 

2. Query 1: Asks the prover A for the entire "accepting" local view (i.e., ttq). 

3. Query 2: Ask the prover B for a random location within the local view (i.e., (vtq)^. 

4. Accept if the answer of the prover B is consistent with the answer of the prover A. 

It is an easy exercise to check the following two facts. The constructed 2-provers PCP has the 
completeness property. Tthe robust soundness of the robust PCP translates into the soundness of 
the 2-provers PCP. A closer look at this transformation reveals that it is in fact, invertible. This 
demonstrates a syntactic equivalence between robust PCPs and 2-prover projection PCPs. Note 
that in this equivalence, the alphabet size of the left prover |E^| translates to query complexity 
of the robust PCP verifier (to be precise, free-bit complexity of robust PCP verifier). Given this 
equivalence, our goal to prove Theorem 5.1.2 can be equivalently stated as constructing for every 
constant 5, robust PCPs for ClRCUlT-SAT with robust soundness 5 and query complexity some 
function of 5 (but independent of n) . 

5.2 Locally Checkable Codes 

Our goal is to construct a robust PCP for the ClRCUlT-SAT over a constant size alphabet with 
constant number of queries for arbitrarily small soundness error. To achieve this goal, we need to 
transform a NP-proof (or a certificate for an NP problem) to a proof that can be locally checked. 
To do this, we use locally checkable codes. There are two potential candidates for locally checkable 
codes. 

1. The first one is the Directed Product code; the new proof of the PCP theorem by Dinur and 
the proof of the parallel repetition theorem of Raz are based on this encoding. 

2. The second one is the Reed-Muller code which is based on the low-degree polynomials over a 
finite field F, and the original proof of the PCP theorem is based on this encoding. 

We use the Reed-Muller code in construction of the robust PCP. 

A PCP, by definition, is a locally checkable encoding of the NP witness. In the rest of today's 
lecture, we shall construct locally checkable encodings of two very specific properties, namely "low- 
degreeness" and "being zero on a sub-cube". We will define these properties formally shortly, 
however it is worth noting that neither of these properties is a NP-complete property. In the next 
lecture, we will show how despite this, we can use the local checkability of these two properties to 
construct PCPs for all of NP. 

5.2.1 Reed-Muller Code 

Let F be a finite field, and let V™ be the set of all m-variate polynomials of degree at most d over 
F. The natural way of specifying a function / 6 V™ is to list the coefficients of /. It is easy to 
check that a m-variate polynomial of degree d has ( Tr jj ) coefficients. The Reed-Muller encoding of 
/ is the list of the evaluations of / on all x € F m ; the codeword at the position indexed by x G F m 
has value f(x). The length of this codeword is |F m |. 

This encoding is inefficient but there is an efficient "local test" to find out if a given codeword 
is close to a correct encoding of a low degree polynomial. 
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• Question: Given a function / : F m — >• F, how does one check if / is a Reed-Muller encoding: 
The straightforward way to do this is to interpolate the polynomial and check if it has degree 
at most d. 

• Question: Given a function / : F m — > F, how does one locally check if / is close to a Reed- 
Muller encoding. A test for this purpose was first suggested by Rubinfeld and Sudan [RS96] 
This test is based on the fact that a restriction of a low-degree polynomial (over F m ) to a line 
(or any space with small dimension) is also a low-degree polynomial. 

5.2.2 Low Degree Test (Line-Point Test) 

Given the evaluations of function / on all points in F m . Our goal is to check if / is close to a 
m-variate polynomial of degree at most d; we do this by checking the values of the function / on 
a random line. A set {x + ty\t G F}, for some x, y G F m , is called a line in F m . 
Low Degree Test (LDT): 

1. Pick a random line t in F m ; this can be done by picking two random points x,y G F m . 

2. Query the function / on all points of the line t. Let f\e denote the restriction of / on the line 
£(i.e.,fU(t) = f(x + ty)). 

3. Accept if f\i is an univariate low-degree polynomial (i.e., f\i G V\). 

Clearly, if / G "P™, then f\i is an univariate polynomial of degree at most d. Hence, we have 
the perfect completeness. 

Completeness: / G V™ =^ Prob [LDT accepts] = 1 

Rubinfeld and Sudan [RS96] proved the following form of soundness for this test. 

Soundness: V<5, 35' : Prob [LDT accepts] > 1 — <5 / is (1 — <5')-close to some low-degree polyno- 
mial (i.e., &gr(f,V™) >l-5'). 

We will actually need the following stronger soundness that was proven by Arora and Su- 
dan [AS03]. 

Stronger Soundness: E[agr(/|^, V^)] > 5 =>■ agr(/, V™) > 8 — me, where e = poly(m, d, tL). 

Raz and Safra [RS97] proved an equivalent statement (with better dependence of |F| on d) for the 
plane-point test as opposed to the line-point test. 

5.2.3 Zero Sub- Cube Test 

In this section, we introduce another test that is used in the construction of robust PCPs. Let / be 
a polynomial over F m and let H be a subset of F. We want to test if / is a low degree polynomial 
(i.e., / G VX) an d if it is zero on the sub-cube H m (i.e., f\n m = 0). Using the low degree test 
(LDT) we can check if / G V™, but to test if / is zero on H m it is not enough to pick few random 
points from H m and test if / is zero on those points. 

Before describing the correct test, we present two results about the polynomials. 



32 



Lemma 5.2.1 (Schwartz-Zippel). Let f be a m-variate polynomial of degree d over ¥ m . If f is 
not a zero polynomial (i.e., f ^ 0), then 



Prob [/(x) = 0] < 



d 



~ IF 



The above lemma shows that if a low degree polynomial over a sufficiently large field is not zero 
at every point, then it can only be zero on small fraction of points. 

Proposition 5.2.2. Let f be a polynomial of degree at most d over¥ m . The restriction of f to 
H m is a zero polynomial (i.e., f\n m = 0) if and only if there exist polynomials q\, . . . , q m of degree 
at most d — \H\ such that 



where gji(x) = YlheH ( x ~ ^ s an univariate polynomial (of degree \H\). 

Now we describe the Zero Sub-cube Test. In the LDT we assumed that the evaluation of / on 
all points are given in the proof table. By the above Proposition if the polynomial / of degree at 
most d is zero on H m then there are polynomials qi, . . . , q m of degree at most d — \H\ that satisfy 
Equation (5.2.1). In the Zero Sub-Cube Test, we require that the proof table also contains the 
evaluations of q±, . . . , q m (in addition to the evaluation of /) on all points in F m . 
Zero Sub-cube Test: 

1. Choose a random line £ in F TO . 

2. For /, 

q\i ■ ■ ■ j qm check if /|^, qi\g, . . . , q m \e is a low degree polynomial. In more detail, check 
if f\i has degree at most d, and for each i = 1, . . , ,m check if q^i has degree at most d— \ H\. 

3. For each x G £, check if f(x) = YaLi 9H{xi)qi{x). 

4. Accept if each of the above tests passes, and reject otherwise. 

Combining the soundness of the low-degree test and the above properties of polynomials, we 
can prove the following completeness and soundness of the Zero Sub-cube Test. Let Z™ denote 
the set of m-variate polynomials P of degree d such that P\H m = 0. Also for any line £, let &cc(£) 
denote the set of accepting local views of the Zero Sub-cube Test for the random line t. 

Completeness: If / G Z™, then Prob [Zero Sub-cube Test accepts] = 1 or equivalently Prob [(f\e, qi\l, 



Soundness: E[agr((/|£, qi\e, . . . , q m \t), eucc(£))] > 5 agr(/,Z^) > 5 - me - d/\¥\, where e = 
poly(ra,d, ||y ). 



m 




(5.2.1) 



i=l 



1. 
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6.1 Recap from Part 1 

Recall that we want to construct a robust PCP for the NP-Complete problem. I.e. for every 
n-sized instance x of the NP-complete problem L we want to construct a proof II, which can be 
checked by a verifier using a random string R of length logn and a constant-size query Q. The 
verifier computes a local predicate (f> of the local view II|q and accepts iff 0(II|q) = 1. We want 
the construction to satisfy the following properties. 

Completeness: If x G L then there exists a proof II such that 

Prob[0(n| Q ) = l] = l. 

.ft 

Soundness: If x £ L then for all proofs II, 

E[agr{U\ Q ,sat{<p))} < 5. 

Recall further that in Part I we constructed a PCP with the parameters above not for any 
NP-complete property but for the specific "Zero on a Subcube" property. We say that a function 
/ : F m ->• F satisfies the "Zero on Subcube" property iff: 

• / is a low-degree polynomial P. 

• P vanishes on H m , where H C F. 
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6.2 Robust PCP for CIRCUIT-SAT 



In this part of the proof we will show how to use the local test for Zero on a Subcube to construct 
a PCP for the CIRCUIT-SAT problem. 

6.2.1 Problem Definition 



1 




Figure 6.1: CIRCUIT-SAT's input 
CIRCUIT-SAT is the following decision problem: 

• Input: A circuit C with n gates (Figure 6.1); k of them are the input gates w\, . . . , Wk, and 
the rest are OR and NOT gates with fan in at most 2 and fanout at most 1. Let's associate 
variables zi,...,z n with each gate (including the input gates) . Variable Z\ is the output of 
gate i. The output gate outputs 1. 

• Output: 1 iff there exists an assignment to z%, . . . , z n that respects the gate functionality, 
and otherwise. 

Note that a proof for this problem is an assignment to z±, . . . , z n , and verifying the proof amounts 
to checking that the assignment respects gate functionality at each gate. To use our local Zero on a 
Subcube test for CIRCUIT-SAT we need to encode the assignment and the circuit C algebraically, 
so that an assignment satisfies C iff a related function is a low-degree polynomial that vanishes 
on a small subcube. Representing the assignment and the circuit algebraically is performed by a 
process known as arithmetization. 

6.2.2 Arithmetization of the Assignment 

First we will map an assignment to the gate variables z\ , . . . , z n to a low-degree polynomial over 
an arbitrary field F m so that the assignment is encoded by the polynomial. 

Let \H m \ = n and choose an arbitrary bijection H m -R- [n]. The assignment maps each gate to 
either or 1, so it is equivalent to a function A : H m — > {0, 1}. We choose H so that {0, 1} C H C F, 
and we can write A : H m — > F. 

The following (easy-to-prove) algebraic fact will be used in the arithmetization of the circuit. 
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Fact 6.2.1 (Low-Degree Extension (LDE)). For any function S : H m —> ¥, there exists a poly- 
nomial S : ¥ m — > ¥ such that S\n m = S and the degree of S for each variable is at most \H\. 
Therefore the total degree of S is at most m\H\. 

Then A : H m — > ¥ is mapped by the low-degree extension to a polynomial A : ¥ m — > ¥ and the 
degree of A is at most m\H\. 

6.2.3 Arithmetization of the Circuit 

Our goal is to derive a rule from the circuit C which maps any polynomial A : F m — > ¥ to a 
different low-degree polynomial P% : F 3m+3 — > F, such that P^|^3m+3 = if and only if A encodes 
a satisfying assignment. Note that the existence of such a rule is all we need to construct a PCP 
for CIRCUIT-SAT, as it reduces verifying a satisfying assignment to testing the Zero on a Subcube 
property. 

We will specify the circuit in a slightly different fashion to enable the arithmetization. Consider 
a function C : H 3m x H 3 — > {0, 1} that takes three indexes h,i2,i3 £ [n] = H m and three bits 
^1)^2,^3 ^ {Oil} H and outputs a bit as follows based on the functionality of the gate whose 
input variables are z% x and Zi 2 and output variable is z% z . 

1, iff the assignment = b±, Zi 2 = hi Zi 3 = 63, where i\ and 12 
are input values to gate is and i% is the output value is an 
INVALID configuration for the gate 13 

otherwise. 

Figure 6.2 illustrates the meaning of the arguments of C. 

z3 = ~b3 
i3 




il \2 
zl = ~bl z2 = ~b2 

Figure 6.2: Setting of gate variables for C 

Now once again we can use the LDE to map C : H 3m +3 _^ jp ^ Q a low-degree polynomial 
C : F 3m+3 -> F. 



C(i 1 ,i 2 ,i3,h,b2,b 3 ) = < 
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We are ready to construct the rule we need. Given any q : F m — > ¥ we define P^ : F 3m+3 — > ¥ 
such that 

P(a) (p'l ! • • • ) ^mi > • ■ • > %2mi %2m+l > ■ ■ • > ^3m; #1 j -^2; Z 3 ) 
v v ' s v ' s v ' 

Xl X2 X3 

= C , (x 1) x 2 ,x 3 , 2 3 )(g(xi) - zi)(g(x 2 ) - z 2 )(g(x 3 ) - 23). 

Note that if g is low-degree, is also low-degree. 

The motivation for defining P^ in this way will become clear when we apply the definition to 

A: 

P {A) (h, i 2 , i 3 , 61, b 2 , 63) = i 2 , is, 61 , b 2 , 6 3 )(A(ii) - 6i)(i(i 2 ) - 6 2 )(i(i 3 ) - 63). (6.2.1) 
It is now an easy case-analysis to observe the following. 
Observation 6.2.2. P,£,\ H 3m+3 = <^ A is a satisfying assignment. 

6.2.4 The PCP Verifier 

Given a circuit C, the PCP proof consists of the oracles A : ¥ m — > ¥ and Pi : F 3m+3 — > ¥. 
The PCP verifier needs to make the following checks: 

• A satisfies the low-degree test 

• Pjt satisfies the low-degree test 

• (P^,A) satisfies the rule described in (6.2.1). 

• P% is zero on the subcube H m . 

Given the low-degree test and zero-on-subcube test, it is straightforward to design a PCP that 
performs the above tests. The PCP verifier expects as proofs the oracles A : ¥ m — > F, Pt : F 3m+3 — >• 
F, qi : F 3m+3 — >• F, . . . , g 3m + 3 : F 3m+3 — > ¥. The oracles qi, . . . , q^m+z are the auxiliary oracles for 
performing the zero-on-subcube test. The verifier first picks a random line £ in IF 3m + 3 . It reads 
the value of all the oracles along the line £. It checks that the restrictions of all the oracles to the 
line is low-degree. It then checks that for each point x on the line I, the "zero-on-subcube" test is 
satisified, namely 

3m+3 

P a( X ) = <li( X )9H{xi). 
i=l 

It finally checks for each point on that (6.2.1) is satisified. This completes the description of the 
PCP verifier. 

For want of time, we will skip the analysis of the Robust PCP (see [BGH + 06] and [Har04] for 
details). 

Let us now compute the parameters of the PCP verifier. Here n = H m is the input length. 
Let us assume m = log(n)/ loglog(n). We can choose |F| = poly(m|i7|). The PCP verifier makes 
0(|F|) = polylogn queries and the amount of randomness used is 0(mlog(|F|)) = O(logn). The 
above construction yields a robust PCP of the following form 
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Theorem 6.2.3. CIRCUIT-SAT has a robust PCP that uses 0(log n) randomness, makes poly log n 
queries and has (1/polylognJ robust soundness parameter. 

Observe that the robust PCP constructed in the above theorem has polylog query complexity 
and not constant, as we had originally claimed. In the next section, we give a high level outline of 
how to reduce number of queries (from polylog to constant) using a composition technique originally 
designed by Arora and Safra [AS98]. 

6.2.5 PCP Composition 

In this section, we describe briefly the PCP composition (due to Arora and Safra [AS98]) that 
helps in reducing the number of queries made by the verifier form polylog(n) to constant without 
affecting the other parameters too much. Recall that the PCP verifier on input x and an oracle 
access to a proof (j>, tosses some random coins and based on the randomness queries some locations. 
Denote the query locations as the set /. The PCP verifier evaluates a CIRCUIT-SAT predicate 
0(Il|j) and accepts or rejects based on the outcome of the predicate. The PCP constructed in the 
previous section achieves 0(log(n)) randomness and 0(poly log(rt)) number of queries. We would 
like to reduce the query complexity from poly log n to 0(1). How do we do this? How does one 
check that n|/ satisfies <p without reading all of n|/ and only reading a constant number of locations 
in n|/. Arora and Safra suggested that use another PCP to recursively perform this check: (p\j = 1. 
Let us denote the original PCP verifier as the outer verifier and the one that checks the predicate <j) 
without reading the entire set I as the inner verifier. The inner verifier gets the circuit 4> an d (n|/) 
as inputs. But if it reads all the input bits then the query complexity is not reduced. Instead, the 
inner gets as input the circuit and an oracle to the proof of and it now needs to check that 

the proof in this location satisfies 0. This requires some care as a simple recursion will not do the 
job. A composition in the context of robust PCPs (or equivalently 2-query projective PCPs) was 
first shown by Moshkovitz and Raz [MR08]. A more generic and simpler composition paradigm 
for was then shown by Dinur and Harsha [DH09]. For want of time, we will skip the details of the 
composition and conclude on the note that applying the composition theorem of Dinur and Harsha 
to the robust PCP constructed in Theorem 6.2.3, one can obtain the constant query PCP with 
arbitrarily small error as claimed before (see [DH09] for the details of this construction). 
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Previously, we saw the proof of the PCP theorem and its connection to proving inapproximability 
results. The PCPs that we have seen so far have constant number of queries, but over a large 
alphabet. Now we are interested in designing useful PCPs while keeping the number of query bits 
low. The purpose of this lecture is to present such a PCP construction, which leads to optimal 
inapproximability results for various problems such as MAX-3SAT and MAX-3LIN. 

7. 1 Introduction 

The PCP theorem and Raz's parallel repetition theorem [Raz98] give the NP-hardness of a problem 
called LABEL COVER (which we will define shortly). This problem is the canonical starting point 
for reductions that prove inapproximability results. Such reductions can be broadly categorized 
into two: 

Direct reductions. These have been successful in proving inapproximability for network prob- 
lems, lattice based problems, etc. 

Long code based reductions. Salient examples of such results can be found in the paper of 
Bellare, Goldreich, and Sudan [BGS98] and Hastad [HasOl]. 

Long code based reductions have been successful for many important problems like MAX-CUT, 
albeit with a catch: many of these results depend on conjectures like the Unique Games Conjec- 
ture [Kho02], which will be the subject of the next lecture. The focus of this lecture is to prove the 
powerful result of Hastad's, which can be stated as follows: 
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Theorem 7.1.1 (Hastad's 3-bit PCP [HasOl]). For every e,i] > 0, NP has a PCP verifier that 
uses O(logn) random bits, queries exactly 3 bits to the proof, evaluates a linear predicate on these 
3 bits, and has completeness 1 — e and soundness 1/2 + rj. 

We can view the bits in this PCP proof as boolean variables. Then the test of the verifier can 
be interpreted as a system of linear equations, each equation corresponding to the triplet of bits 
tested for a given random string. Thus, we immediately obtain the hardness for the MAX-3LIN 
problem (where an instance of MAX-3LIN is a system of linear equations modulo 2 with at most 
3 variables per equation, and we are interested in maximizing the fraction of equations that can 
simultaneously be satisfied). 

Corollary 7.1.2. For every e,rj > 0, given an instance of MAX-3LIN, it is NP-hard to tell ifl — e 
fraction of the equations are satisfiable by some assignment or that no assignment satisfies more 
than 1/2 + i] fraction of the equations. 

In other words, getting an efficient algorithm with approximation guarantee better than (1/2 + 
Tj)/(1 — e), which is ~ 1/2 (since we can choose e and r\ to be arbitrarily small), for MAX-3LIN 
is impossible unless P = NP. This result is tight since a random assignment satisfies half of the 
equations in expectation. Note that the imperfect completeness in the result is essential if P ^ NP, 
since Gaussian elimination can be used to efficiently check whether any system of linear equations 
can be completely satisfied. 

We now go on to the proof of Theorem 7.1.1. Here, the concepts of Proof composition, Long 
codes and Fourier analysis play a pivotal role. We will start with a high-level picture of the PCP 
and work our way down to the actual 3-bit test. 

7.2 Proof Composition 

The standard method to construct a Long code based PCP is by composing an Outer PCP with 
an Inner PCP. These two concepts are explained below. 

The Outer PCP 

The Outer PCP is based on a hard instance of the LABEL COVER problem. 

Definition 7.2.1. A LABEL COVER problem C(G(V,W,E), [m], [n], {tt vw \(v,w) G E}) consists of: 

1. A bipartite graph G(V, W, E) with bipartition V, W. 

2. Every vertex in V is supposed to get a label from a set [m] and every vertex in W is supposed 
to get a label from a set [n] (n > m). 

3. Every edge (v,w) £ E is associated with a projection ir vw : [n] \— > [m]. 

We say that a labeling 4> '■ V \-t [m], (f> : W \-t [n] satisfies an edge (v,w) if tt vw (4>(w)) = 4>(v). The 
goal is to find a labeling that maximizes the number of satisfied edges. 

Let us define opt(£) to be the maximum fraction of edges that are satisfied by any labeling. 
As mentioned earlier, the hardness of LABEL COVER is obtained by combining the PCP theo- 
rem [FGL+96, AS98, ALM+98] with Raz's parallel repetition theorem [Raz98]. 
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Theorem 7.2.2. For every 5 > 0, there exist m and n such that given a LABEL COVER instance 
C(G, [m], [n], {7r V w}), it is NP-hard to tell if opt(£) = 1 or opt(£) < 5. 

It is useful to think of the above theorem as a PCP that makes 2 queries over the constant (but 
large) alphabets of size m and n. The verifier just picks an edge at random, queries the labels of 
the endpoints of this edge and accepts if and only if these labels satisfy the edge. This PCP, which 
is based on a hard instance of LABEL COVER forms our Outer PCP. 

The Inner PCP 

The Outer PCP verifier expects some labels as the answers to its two queries. We will compose this 
verifier with an Inner PCP verifier, which expects the proof to contain some encoding (in our case, 
the Long Code) of the labels, rather than the labels themselves. We choose to have an encoding 
of the labels so that we can check just a few bits of the proof and tell with a reasonable guarantee 
whether the labeling is valid. 

Thus, the Inner verifier expects large bit strings (supposed to be encodings) for each vertex in 
G. Let the edge picked by the Outer verifier be (v, w). Then the Inner verifier checks 1 bit from g v 
(the supposed encoding of the label of v) and 2 bits from f w (the supposed encoding of the label of 
w). Note that the Inner verifier is trying to simulate the Outer PCP. It needs to check the following 
two things in one shot: 

Codeword Test The strings f w and g v are correct encodings of some j € [n] and i £ [m\. 
Consistency Test These i and j satisfy ir(j) = i. 

We can therefore convert a PCP which asked 2 large queries to a PCP which asks 3 'bit' queries. 
We now need to show the following two implications: 

1. (Completeness) opt(£) = 1 => 3 Proof Pr[acc] > 1 — e. 

2. (Soundness) opt(£) < S => V Proofs Pr[acc] < \ + rj. 

The completeness follows by the design of the PCP and should be regarded as a sanity check on the 
construction. The soundness will be proved by contraposition. We will assume that Pr[acc] > 1 /2+r] 
and then decode the labels to satisfy a lot of edges. 

7.3 The Long Code and its Test 

Hastad's Inner PCP verifier does the codeword test and consistency test in one shot. For clarity, 
let us first analyze just the codeword test. We will see how to incorporate the consistency test in 
the next section. For the codeword test, we need to look at the particular encoding that the Inner 
PCP will use: the Long Code. It is defined below. 

Definition 7.3.1. The Long Code encoding of j € [n] is defined to be the truth table of the boolean 
dictatorship function on the jth coordinate, f : {±1}™ {±1} such that f(x\, . . . ,x n ) = Xj. 

Some observations are in order. Note that we are representing bits by {±1} and not {0,1}. 
This is done just for the sake of clarity, since the calculations done with {±1} are less messy. Also 
note that the Long Code is huge. An element j £ [n] will require logn bits to represent, but the 
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Long Code of j requires 2 n bits, a doubly-exponential blowup! We can get away with this because 
n, the alphabet size, is a constant. 

Now for a short aside on Fourier analysis. Recall that for each S C [n], the Fourier character 
Xs ■ {±1}™ ^ {±1} is defined as 

Xs(x) = Y{xi. 
ieS 

These characters form an orthonormal basis for the set of Boolean functions / : {±1}™ i— >■ {±1}. 
All such functions can therefore be written in terms of this basis, called the Fourier expansion, as 

f(x) = f(S) X s(x). 

SC[n] 

where f(S) is called the Fourier coefficient of set S. For Boolean functions, these coefficients satisfy 
Parseval's identity, namely Ylscin] fi^) 2 = 1. 

Back to the Long Code. In terms of Fourier expansion, the Long Code of element j is the 
same as the function X{j}- Thus it is simple to write down, since the only non-zero coefficient is 
/({j}) = 1. The Long Code then fits into a general class of functions which have high Fourier 
coefficients of low order. This fact will be useful in the soundness analysis of the test. 

On to the codeword test. This will essentially be a linearity test, that is, we will check whether 

f(x + y) = f(x) + f(y). 
Since we are in the {±1} domain, this linearity translates to checking whether 

f(xy) = f(x)f(y). 

We will also introduce a small randomized perturbation in the linearity test. This is done to improve 
the overall soundness of the test. 

Definition 7.3.2. An e -perturbation vector is a string of ±1 bits, where each bit is independently 
set to —1 with probability e and 1 with probability 1 — e. 

The final codeword test is described below. It is a randomized 3-bit linear test that checks 
whether the input function is close to a Long Code. 
Long Code Test: 

Input: Function f : {±l} n {±1} and error parameter e. 

Test: Pick x,y G {±1}™ at random. Pick an e -perturbation vector fj, G {±1}™ and let z = xy.fi. 
Accept if and only if 

f(z) = f(x)f{y). 

Let us analyze the completeness and soundness of this test. We have the following theorem. 

Theorem 7.3.3. Given a truth table of a function f : {±1}™ i-> {±1} and e > 0, the following are 
true for the Long Code Test: 

1. If f = X{j} for some j, then Pr[acc] = 1 — e. 
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2. If Pr[acc] > 1/2 + n, i/ien / "resembles" a Long Code in the following sense: 3 S C [n] 
stjc/i £/iai > and |5| < 0((l/e) log(l/r/)) ; m other words, there exists a large Fourier 

coefficient of low order. 

Proof. To prove the completeness part, assume that / = XU\ f° r some j £ [n], that is, it is a Long 
Code. Then the test will "Accept" if and only if 



Zj — Xjyj < — > XjUj^j — Xji/j 



M = 1 



which happens with probability 1 — e. 

For the soundness analysis, assume that Pr[acc] > 1/2 + rj. We can write this probability in 
terms of the test as 

This is a standard PCP trick that is used often in analyzing such tests. Substituting for Pr[acc] 
and using the Fourier expansion of /, 

l + V<l + l®x,yAf(x m )f(x)f(y)} 



2r) < E x . 



E n s )xs(x W ) e f( T M 



iSC\n] 



vTCW 



,l/C[n] 



E f(S)f(T)f(U)E w [xs{xy^)xT{x)xu{y)] 

?,T,C7 

E f(S)f(T)f(u)E x ^ [xs(x)xs(y)xs(»)xT(x)xu(y)} 

?,T,C7 

E f(S)f(T)f(u) e x Ixs(x)xt(x)} E y lxs(y)xu(y)} E, \xs(l*)] ■ 



S.T.U 



We can simplify the last expression by using orthonormality of the x' s to argue that 



E»[xsMxtW] = E a 



ieSAT 



1 if S = T, 
otherwise 



where SAT stands for the symmetric difference between sets S and T. Thus the terms in the 
summation will vanish unless S = T = U . We then get 



2r?<E/(S) 3 lMxs(/^- 

s 



(7.3.1) 



As an aside, the Long Code Test without perturbations was analyzed long before Hastad by Blum, 
Luby and Rubinfeld [BLR93]. In that case, we just get 

E f(Sf > 2r? => liLxl E f( S f > 2r? ^ 3 large \f\ (since E f = 1 )- 
s s 

Now, with perturbation, we have 



E M [xs(fJi)} = E, 



[1(1 - e) + (-l)e] 151 = (l-2e)l 5 L 
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Substituting this in Inequality (7.3.1), we get 

2 V <Y,f(S)\l-2ef\. 

s 

We can again think of the above sum as a convex combination (since ^ J 2 = 1). This implies that 
there exists an S such that /(5)(1 — 2e) ' s '' > 2n. Thus we have 

Pr[acc] >\+V =>- 3 5: \f(S)\ > 2rj and \S\ < O Q log M . □ 

The Long Code test can be thought of as an analog of the concept of gadget in NP reductions. 
In particular, Theorem 7.3.3 is very important and is the crux of the PCP. We will prove similar 
results in the analysis of later tests. 

7.4 Incorporating Consistency 

Let us restate what we want from our 3-bit test. Recall that the inputs to the Inner PCP verifier 
are two supposed Long codes, g and /, of two vertices v and w, and the projection ir between them. 
We have to check two things in one shot: 

1. g and / are Long codes of some i E [m] and j E [n]. 

2. 7T(i) = i. 

We are going to do this by reading 1 bit from g and 2 bits from / and applying a 3-bit linear test 
similar to the Long code test. 
Consistency Test: 

Input: Functions g : {±l} m 1— >• {±1} and f : {±l} n 1— > {±1}, projection tt : [n] 1— > [m], and 
error parameter e. 

Test: Pick x E {±l} m ; y E {±l} n at random. Pick an e -perturbation vector \x : {±1}™. Let 
z=(io vr)y/u. Accept if and only if 

f(z)=g(x)f(y). 

In the above test, the vector (xott) is defined as (xoir)j = x^u\ V 1 < j < n. Such a definition 
is needed because x and y are vectors of different sizes. 

The analysis of the above test is similar to that of the Long Code test. Hence, we will skip a 
rigorous proof and state only the important details. The completeness is simple to analyze. 

For the soundness analysis, we can imagine a restricted case where n = m, tt = id (identity 
permutation), and / = g. Then the test is exactly the 3-bit Long Code test that we saw in the 
previous section and we get 

r 1^1, Theorem 7.3.3 ?/C<\3\ Cauchy-Schwarz ^ ?7o\4 ^ 2 

Pr[acc] > - + rj ► ^ f( S > ^ V ► 2^ f( S > - ^ ' 

SC[n] SC[n] 
|S|<0(ilogi) |5|<0(Mogi) 
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Now if we have different / and g, one can verify by analysis similar to that in Theorem 7.3.3 that 
instead of the above inequality, we would get 

E 9(Sff(Sf > V 2 . 

SC[n] 
|5[<0(ilogi) 

Further, now if ir ^ id and n ^ m, then we would end up with the inequality in the following 
theorem. 

Theorem 7.4.1. The following are true for Consistency Test(g, f,ir,e): 
1- If f = X{j}, 9 = X{i] and n(j) = i, then Pr[acc] = 1 - e. 
2. //Pr[acc] > 1/2 + rj, then f and g are correlated in the following sense: 

E 9{S?f{T? > r! 2 

SC[m],TC[n] 
|S|,|T|<0(l/elog(l/»,)) 
S ,T correlated by w 

where "S,T correlated by ir" means that there exist i G S , j G T such that 7r(j) = i. 

We are now ready to describe Hastad's composed PCP verifier. 
Hastad's 3-bit PCP: 

Input: Hard instance o/LABEL COVER, C(G{V, W, E), [m], [n], {ir vw }) (given by Theorem 7.2.2) 
and error parameter e. 

Verifier: Pick edge (v, w) G E at random. Let g v and f w be the supposed Long codes of the two 
vertices. Run Consistency Test(g v , f w ,ir VV} , e). 

The following theorem gives the completeness and soundness of the verifier. 

Theorem 7.4.2. Given a hard instance o/LABEL COVER, C, Hastad's PCP guarantees 

1. (Completeness) If opt(C) = 1, then there exists a proof for which Pr[acc] > 1 — e. 

2. (Soundness) //Pr[acc] > 1/2 + 277, then opt(£) > e 2 rj 3 / log 2 (l/ry), that is, there is a labeling 
to £ that satisfies at least e 2 // 3 / log 2 (I/77) fraction of edges. 

Proof. For the completeness part, we can assume that the proof contains correct encodings on 
correct labels. Then by Theorem 7.4.1, the Verifier accepts with probability 1 — e on every choice 
of edge. Therefore, the overall acceptance probability also remains 1 — e. 

Now for the soundness part. If Pr[acc] > 1/2 + 2?y, then by an averaging argument, for at least 
77 fraction of the edges (v, w) G E, Consistency Test^, f w ,iT vw ,e) accepts with probability at least 
l/2 + »7. 

Fix any such good edge. Then by Theorem 7.4.1, we have 

E 9v(S) 2 f w (T) 2 >V 2 - (7.4.1) 

|5[,[T|<0(Velog(l/i7)) 
3 i€S,j£T : Tr(j)=i 

We will now define labels for vertices v and w. First, we pick sets S C [m] and T C [n] with 
probability g v (S) 2 and f w (T) 2 respectively. Next, we pick labels i G S and j G T at random. 
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This is a randomized labeling and by the probabilistic method, the argument goes through. In 
expectation, at least 



2 

*i . ' .*! ' ; — 27" 



log 2 (l/>?) 

Pr[Pick Pr[Pick correlated S,T] s v ' 

good edge] by Inequality 7.4.1 Pr[Pick i£S,j£T 

s.t. n(j)=i] 

fraction of label-cover edges are satisfied. □ 

Finally, observe that if we set 5 = c£ 2 rj 3 / log 2 (l/ry), then Theorem 7.2.2 and Theorem 7.4.2 
together prove Theorem 7.1.1. 



7.5 Concluding Remarks 

In this lecture, we have seen and analyzed Hastad's powerful 3-bit PCP. There are a few subtleties 
in the construction of this PCP that are worth mentioning. The first subtlety, that we have already 
seen in the Introduction, is that we have to sacrifice perfect completeness if we want our Verifier 
to have linear predicates. 

The second issue is that any linear test can be satisfied if everything is (or +1 in our case). 
Moreover, in the soundness analysis of Theorem 7.4.2, the sets S and T that are chosen by the 
probability distributions {g v (S) 2 } and {f w {T) 2 } respectively should not be empty. This problem 
is taken care by an operation called folding. For more information, the reader can look at the 
references below. 

Some good references for more information on Hastad's PCP are Chapter 22 in Arora and 
Barak's textbook [AB09], Khot's article on Long code based PCPs [Kho05] and Hastad's original 
paper [HasOl]. 
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Lecture 8 



Semide finite Programming and Unique Games 
Moses Charikar 

Scribe: Alantha Newman 
21 July, 2009 



8.1 Unique Games 

The topic of Unique Games has generated much interest in the past few years. The Unique Games 
Conjecture was posed by Khot [Kho02]. We will discuss the associated optimization problem and 
the algorithmic intuition and insight into the conjecture, as well as the limits of these algorithmic 
techniques. Finally, we mention the amazing consequences implied for many optimization problems 
if the problem is really as hard as conjectured. 

We now define the Unique Games problem. The input is a set of variables V and a set of k 
labels, L, where k is the size of the domain. Our goal is to compute a mapping, £ : V — > L, satisfying 
certain constraints that we now describe. Let E denote a set of pairs of variables, {{u, v)} C V x V. 
For each (u,v) £ E, there is an associated constraint represented by ir uv , indicating that £{v) 
should be equal to 7r uv (£(u)); we assume that the constraint the inverse of the constraint tTvu 

i.e, tt uv = 7r~J. Thus, our goal is to compute the aforementioned mapping, £ : V — > L, so as to 
maximize the number of satisfied constraints. 

Each constraint, tt uv , can be viewed as a permutation on L. Note that this permutation may 
be different for each pair (u, v) E E. For a pair (u, v) G E, if v is given a particular label from 
L, say £(v), then there is only one label for u that will satisfy the constraint tt uv . Specifically, 
£{u) should equal ir vu (£(v)). Hence, the "unique" in Unique Games. The practice of calling this 
optimization problem a unique "game" stems from the connection of this problem to 2-prover 1- 
round games [FL92]. The Unique Games problem is a special case of Label Cover (discussed in 
other lectures in the workshop), in which each constraint forms a bijection from L to L. Having 
such a bijection turns out to be useful for hardness results. 
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8. 2 Examples 



We will refer to E as a set of edges, since we can view an instance of Unique Games as a graph 
G = (V,E) in which each edge (u,v) £ E is labeled with a constraint ir uv . We now give some 
specific examples of optimization problems that are special cases of Unique Games. 

8.2.1 Linear Equations Mod p 

We are given a set of equations in the form Xi — Xj = Cij (mod p). The goal is to assign each 
variable in V = {xi} a label from the set L = [0, 1, . . .p — 1] so as to maximize the number of 
satisfied equations. Note that each constraint is a bijection. 

8.2.2 MAXCUT 

Given an undirected graph G = (V, E), the Max Cut problem is to find a bipartition of the vertices 
that maximizes the weight of the edges with endpoints on opposite sides of the partition. 

We can represent this problem as a special case of Linear Equations mod p and therefore as a 
special case of Unique Games. For each edge (i, j) E E, we write the equation xi — Xj = 1 ( mod 2). 
Note that the domain size is two, since there are two possible labels, and 1. 

8.3 Satisfiable vs Almost Satisfiable Instances 

If an instance of Unique Games is satisfiable, it is easy to find an assignment that satisfies all of 
the constraints. Can you see why? Essentially, the uniqueness property says that if you know 
the correct label of one variable, then you know the labels of all the neighboring variables. So 
we can just guess all possible labels for a variable; at some point your guess is correct and this 
propagates correct labels to all neighbors, and to their neighbors, and so on. This is a generalization 
of saying that if a graph is bipartite (e.g. all equations in the Max Cut problem are simultaneously 
satisfiable), then such a bipartition can be found efficiently. So when all constraints in an instance 
of Unique Games are satisfiable, this is an "easy" problem. 

In contrast, the following problem has been conjectured to be "hard": If 99% of the constraints 
are satisfiable, can we satisfy 1% of the constraints? The precise form of the conjecture is known as 
the Unique Games Conjecture [Kho02]: For all small constants e, S > 0, given an instance of Unique 
Games where 1 — e of the constraints are satisfied, it is hard to satisfy a 5 fraction of satisfiable 
constraints, for some k > f(e, 5), where k is the size of the domain and / is some function of e and 
5. 

How does / grow as a function of e and <5? We claim that f(e, 5) > 1/5. This is because it can 
easily be shown that we can satisfy a 1/k fraction of the constraints: Randomly assigning a label to 
each variable achieves this guarantee. Thus, in words, the conjecture is that for a sufficiently large 
domain size, it is hard to distinguish between almost satisfiable and close to unsatisfiable instances. 

8.3.1 Almost Satisfiable Instances of MAXCUT 

We can also consider the Max Cut problem from the viewpoint of distinguishing between almost 
satisfiable and close to unsatisfiable instances. However, for this problem, a conjecture as strong as 
that stated above for general Unique Games is clearly false. This is because we can always satisfy 
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at least half of the equations. (See Sanjeev's lecture.) We now consider the problem of satisfying 
the maximum number of constraints given that a (1 — e) fraction of the constraints are satisfiable. 
We write the standard semidefinite programming (SDP) relaxation in which each vertex u (with a 
slight abuse of notation) is represented by a unit vector, u. 

El — u ■ v 

2 

(u,v)eE 
u ■ u = 1 \/u G V 
u G M n Vu G V. 

For a fixed instance of the Max Cut problem, let OPT denote the fraction of constraints satisfied 
by an optimal solution, and let OPTsdp denote the value of the objective function of the above 
SDP on this instance. If OPT > (l-e)\E\, then OPT SD p > (l-e)\E\, since OPT SD p > OPT. In 
Lecture 1 (Sanjeev's lecture), it was shown that using the random hyperplane rounding of Goemans- 
Williamson [GW95], we can obtain a .878- approximation algorithm for this problem. We will now 
try to analyze this algorithm for the case when OPT is large, e.g. at least (1 — ff)|-E7|. From a 
solution to the above SDP, we obtain a collection of n-dimensional unit vectors, where n = \ V\. We 
choose a random hyperplane, represented by a vector r G A^O, l) n (i.e. each coordinate is chosen 
according to the normal distribution with mean and variance 1). Each vector u £ V has either a 
positive or a negative dot product with the vector r, i.e r ■ u > or r ■ u < 0. Let us now analyze 
what guarantee we can obtain for the algorithm in terms of e. 

As previously stated, we have the following inequality for the SDP objective function: 

(u,v)eE 

Let 6' uv represent the angle between vectors u and v, i.e. arccos(u • v). Let 6 UV denote the angle 
(tt — 6' uv ). Then we can rewrite the objective function of the SDP as: 

E l + cos{9 uv ) 
2 

(u,v)eE 



Further rewriting of the objective function results in the following: 

El + COS(0 UV ) _ 1 - cos (9 UV ) 

2 _ J 2 

(u,v)eE (u,v)eE 

\E\ - V" 1 ~ cos (9 UV ) 
(u,v)eE 

= \E\- £ sin 2 (^) 

(u,v)GE 

> \E\-s\E\. 

We say that vertices u and v are "cut" if they fall on opposite sides of the bipartition after rounding. 

Pr[u and v cutj = = 1 . 

7T TT 
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The expected size of S — the number of edges cut in a solution — is: 

E[S] = Yl 1 ~ — 

(u,v)eE 

= \E\ - V — 

(u,v)eE 

Assume for all (u,v) G E that sin 2 {^f-) = e. Then sin(^f) = sje. For small 9, we have that 
sin (9) « 9. Therefore, 9 uv /2 « y/e. 

Thus, the expected value EfS] > \E\ {1—cyfe) for some constant c. In other words, if we are given 
a Max Cut instance with objective value (1 — we can find a solution of size (1 — Cy/e)\E\. In 

other words, an almost satisfiable instance can be given an almost satisfying assignment, although 
the assignment has a weaker guarantee. 

8.4 General Unique Games 

What happens for a large domain? How do we write an SDP for this problem? Before we had 
just one vector per vertex. Now for each variable, we have k values. So we have a vector for each 
variable and for each value that it can be assigned. First, we will write a {0, 1} integer program 
for Unique Games and then we relax this to obtain an SDP relaxation. 

8.4-1 Integer Program for Unique Games 

Recall that L is a set of k labels. For each variable u and each label ieL, let U{ be an indicator 
variable that is 1 if u is assigned label i and otherwise. Note that the expression in the objective 
function is 1 exactly when a constraint tt uv is satisfied. 

max Yj J2 Ui ' 

(u,v)eE ieL 

J^Ui = 1 \/u G V. 

ieL 

Now we move to a vector program. The objective function stays the same, but we can add some 
more equalities and inequalities to the relaxation that are valid for an integer program. Below, we 
write quadratic constraints since our goal is ultimately to obtain a quadratic program. 

Ui • Ui = 1 Vu G V, i G L, 

ieL 

Ui - Uj = Vu G V, i 7^ j G L. 

Additionally, we can also add triangle- inequality constraints on triples of vectors, {ui, Vj, Wh} for 
u, v, w G V and i, j, h G L: 

\\ui-w h \\ 2 < \\ui-Vj\\ 2 + \\vj - w h \\ 2 , (8.4.1) 

Ih-^li 2 > \\ui\\ 2 - \\vj\\ 2 . (8.4.2) 

These constraints are easy to verify for 0/1 variables, i.e. for integer solutions. Note that these 
constraints are not necessary for the integer program, but they make the SDP relaxation stronger. 
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8-4-2 Trevisan's Algorithm 

We now look at an algorithm due to Trevisan [Tre08] . Recall that if we know that every constraint 
in a given instance is satisfiable, then we can just propagate the labels and obtain a satisfiable 
assignment. The algorithm that we discuss is roughly based on this idea. 

How can we use a solution to the SDP relaxation to obtain a solution that satisfies many 
constraints? Suppose that OPT is \E\ and consider two vertices u and v connected by an edge. In 
this case, the set of k vectors corresponding to u is the same constellation of k vectors corresponding 
to vertex v, possibly with a different labeling. If OPT is (1 — then although these two 

constellations may no longer be identical, they should be "close" . The correlation of the vectors 
corresponds to the distance, i.e. high correlation corresponds to small distance. Thus, we want to 
show that the vector corresponding to the label of the root vertex r is "close" to other vectors, 
indicating which labels to assign the other vertices. 

An Algorithm for Simplified Instances 

Consider the following "simplified instance" . Recall that the constraint graph consists of a vertex 
for each variable and has an edge between two variables if there is a constraint between these 
two variables. Suppose the constraint graph has radius d: there exists a vertex r such that every 
variable is a distance at most d from vertex r. The following lemma can be proved using the ideas 
discussed above. 

Lemma 8.4.1. If every edge contributes 1 — e/8(d+l) to the SDP objective value, then it is possible 
to efficiently find an assignment satisfying a (1 — e)-fraction of the constraints. 

We now give the steps of the rounding algorithm. 



Rounding the SDP 




(i) Find root vertex, r, such that every 


other vertex is reachable from r by a path 


of length at most d. 




(ii) Assign label i to r with probability 


INI 2 - 


(iii) For each u G V, assign u label j, 


where j is the label that minimizes the 


quantity \\uj — r i 2 . 





As mentioned earlier, the intuition for this label assignment is that Uj is the vector that is 
"closest" to rj. We now prove the following key claim: For each edge (u, v), the probability that 
constraint tt uv is satisfied is at least 1 — e. In particular, recall that edge (u, v) is mislabeled if 
£(v) ^ ir uv (£(u)). Thus, we want to show that the probability that edge (u,v) is mislabeled is at 
most e. 

Since r is at most a distance d from all other vertices, a BFS tree with root r has the property 
that each u has a path to r on the tree of distance at most d. Fix a BFS tree and consider the 
path from r to ti: r = u ,^ 1 ,^ 2 , . . . , u' _1 ,u* = u, where t < d. Let 7r n i denote the permutation 
n u o u i, and recursively define ir u k as the composition of permutations {-K u k u k-i) ■ (vr u fe-i). Let 

= (tTmd) • (vr u ). We now compute the probability that vertex u is assigned label ~n u ^) and that 
vertex v is assigned label tt v u\, given that r is assigned label i. Note that if both these assignments 
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occur, then edge (it, v) is satisfied. (Since edge (it, v) may also be satisfied with another assignment, 
we can think of our calculation as possibly being an underestimate on the probability that edge 
(u, v) is satisfied.) 

Let A(u) denote the label assigned to vertex u by the rounding algorithm. We will show: 

Pr[A(«) = 7r„(t)] > 1-| and Pr[A(«) = 7r„(t)] > 1 - |. 

This implies that the probability that constraint 7r n „ is satisfied is at least 1 — e. Now we compute 
the probability that A(u) ^ TT u (i). Suppose that Uj for j ^ ir u (i) is closer to vector ri than u v u\ 
is. In other words, suppose: 

\\uj-n\\ 2 < \\un u (id -nW 2 - (8.4.3) 

Let B u be the set of labels such that if r is assigned label i £ B u , then it is not assigned label ir u (i). 
Note that label j belongs to B u iff inequality (8.4.3) holds for j. Thus, the probability that it is 
not labeled with 7r u (t) is exactly: 

|2 



PrL4(u) ^ 7r„(i)] = E 



r 



iG-B u 



One can verify that if there is some label j such that inequality (8.4.3) holds, then the quantity 
||rj|| 2 is at most 2||rj — u^^H 2 . This proof makes use of inequalities from the SDP, (8.4.1) and 
(8.4.2), as well as inequality (8.4.3). (See Lemma 8.6.1 from [Tre08], which we include in the 
Appendix.) Recall that each edge in the graph (and thus each edge on the path from r to u in the 
BFS tree) contributes at most 1 — e/8(d + 1) to the objective value. By triangle inequality, this 
implies that X^ieL ll r » — n 7r u («)H 2 — e /^' Thus, we conclude: 

Pr[A(u) + 7r„(i)] = ^INI 2 

< 2 ll r i-^u(i)H 2 

< z^Wn - u nu{j) \\ 2 



< 



£ 

2' 



Similarly, we conclude that Pr[A(w) 7^ 7r v (i)] < e/2, which implies that the probability that con- 
straint tt uv is not satisfied is at most e. 



Shift Invariant Instances 

In the case of Linear Equations mod p, we can add more constraints to the SDP relaxation, which 
allow for a simplified analysis of the rounding algorithm. For any assignment of labels, we can shift 
each of the labels by the same fixed amount, i.e, by adding a value k £ L to each label, and obtain 
an assignment with the same objective value. This property of a solution has been referred to as 
shift invariance. In these instances, the following are valid constraints. Note that p = \L\. 

, ,,2 1 

\ui\\ = - u S V, i G L, 
p 

Ui ■ vj = u i+k ■ v j+k u, v e V, i, j, k <E L. 
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In this case, we obtain a stronger version of Lemma 8.4.1. 



Lemma 8.4.2. In a shift invariant instance in which every edge contributes more than 1— l/2(d+l) 
to the SDP objective value, it is possible to efficiently find an assignment that satisfies all of the 
constraints. 

We will show that in this case, the vector n is closer to vector u 7Tu ^ than to vector uj for any 
label j vr u (i). In other words, rj • u n /{\ > ri ■ Uj for all j £ L. If each edge contributes more than 
1 — l/2{d + 1) to the objective value, then | \ri — u %u ^\ | 2 < I /p. This implies that • u 7Tu ^ > l/2p. 
By triangle inequality, we have: 

1 1 2 i II 1 1 2 

■n\\ + \\n- 
l 

2n ■ uj + - =^ 
p 



Assuming that vector Uj is closer to rj than vector u n tj\, we obtain the following contradiction: 

1 1 

2p WuW ~ 3 ~ 2p 

Note that in the case of shift invariance, r is assigned each label from L with equal probability. 
Because of shift invariance, it does not actually matter which label r is assigned. Thus, we can just 
assign r a label i arbitrarily (we no longer need randomization) and then proceed with the rest of 
the SDP rounding algorithm. 



U TT u (i)\\ 2 


< 


IK ■ 


2 




2 




< 




p 




P 






1 


n ■ Uj 


< 


2p- 



Extension to General Instances 

Applying this SDP rounding to general graphs may not yield such good results as in Lemmas 8.4.1 
and 8.4.2, since the radius of an arbitrary graph can be large, and the objective values of the SDP 
relaxation would therefore have to be very high for the lemmas to be applicable. In order to apply 
these lemmas, we break the graph into pieces, each with a radius of no more than 0(log n/e). Doing 
this requires throwing out no more than an e-fraction of the constraints. The following lemma is 
originally due to Leighton and Rao [LR99] and can also be found in [Tre08]. 

Lemma 8.4.3. For a given graph G = (V, E) and for all e > 0, there is a polynomial time 
algorithm to find a subset of edges E' C E such that \E'\ > (1 — e)|-E|, and every connected 
connected component of E' has diameter 0(log \E\/e). 

Using this lemma, we obtain the following guarantee for general instances: Given an instance 
for which OPT is at least (1 — ce 3 / log n)\E\, we can efficiently find a labeling satisfying a 1 — e 
fraction of the constraints. Note that c is an absolute constant. For shift invariant instances, we 
can satisfy (1 — e)\E\ of the constraints for an instance where OPT is at least (1 — ce 2 / log n)\E\. 

Given a graph, we remove the | fraction of constraints that contribute the least to the objective 
value. This leaves us with at least (1 — e/3)|S| constraints that each contributes at least 1 — 
3ce 2 /logn (or 1 — 3ce/logn for shift invariant instances) to the objective value. We can apply 
Lemma 8.4.1 (or Lemma 8.4.2) with d = logn/e, satisfying at least (1 — 2e/3)\E\ constraints (or 
(1 — e/3)\E\ constraints). 
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8.5 Improving the Approximation Ratio 



Algorithms with improved approximation guarantees for Unique Games have been presented in 
[GT06a, CMM06]. The latter work gives an algorithm with the following guarantee: Given an 
instance of Unique Games with a domain size k for which OPT is at least (1 — e)|-£7|, the algorithm 
produces a solution that satisfies at least max{l — \fe log k, /c~ £ /( 2_£ )} fraction of the constraints. 
Furthermore, it has been shown that the existence of an efficient algorithm that can distinguish 
between instances in which (1 — e)\E\ constraints can be satisfied and those at which less than k~ £ / 2 
constraints can be satisfiable, would disprove the Unique Games Conjecture [KKMO07]. Moreover, 
it is sufficient to refute the conjecture if this algorithm works only for the special case of Linear 
Equations mod p. Thus, focusing on shift invariant instances is a reasonable approach. 

Additionally, the Unique Games problem has been studied for cases in which the constraint 
graph is an expander; in an instance in which OPT is at least (1 — one can efficiently find 

a solution satisfying at least 1 — O(f) fraction of the constraints, where A is a function of the 
expansion of the graph [AKK+08, MM09]. 

8.6 Consequences 

The interest in the Unique Games Conjecture has grown due to the many strong, negative conse- 
quences that have been proved for various optimization problems. Assuming the Unique Games 
Conjecture, it has been shown that the Goemans- Williamson algorithm for Max Cut (presented 
in Sanjeev's lecture) achieves the optimal approximation ratio [KKMO07]. More surprisingly, 
there are many other NP-complete optimization problems for which the best-known approxima- 
tion guarantees are obtained via extremely simple algorithms. Nevertheless, no one has been able 
to find algorithms with improved approximation guarantees, even when resorting to sophisticated 
techniques such as linear and semidefinite programming. Such optimization problems include the 
Minimum Vertex Cover problem and the Maximum Acyclic Subgraph problem, for which the best- 
known approximation factors are 1/2 and 2, respectively. If the Unique Games Conjecture is true, 
then these approximation ratios are tight [KR08, GMR08]. This phenomena has been investigated 
for several other optimization problems as well. A recent result shows that for a whole class of 
constraint satisfaction problems, which can be modeled using a particular integer program, the in- 
tegrality gap of a particular SDP relaxation is exactly equal to its approximability threshold under 
the Unique Games Conjecture [Rag08]. 

Appendix 

We include the following lemma from [Tre08] and its proof: 

Lemma 8.6.1. Let r,u, v be vectors such that: (i) u • v = 0, (ii) ||r — u|| 2 > ||r — v|| 2 , and (in) 
the vectors r,u,v satisfy the triangle inequality constraints from the SDP. Then ||r — u|| 2 > |||r|| 2 . 

Proof. There are three cases: 

1. If||u|| 2 < i||r| | 2 , then by (8.4.2), we have: 

II ||2\||||2 || ||2\ 1 ii ||2 

|| r — u || > || r || — || u || > 2^ r ^ ' 
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2. If||v|| 2 < \ | |r| | 2 , then by (8.4.1), and subsequently (8.4.2), we have: 

llr — u|| 2 > llr — vll 2 > llrll 2 — llvll 2 > -llrll 2 . 



3. If ||u|| 2 , ||v|| 2 > 2l|r|| 2 , then from (8.4.1) and assumption (ii), we have: 

||v-u|| 2 < ||v - r|| 2 + ||r - u|| 2 < 2||r-u|| 2 . 
By Pythagoras theorem and by orthogonality of u and v (assumption (i)), we have: 

II Il2 || \\2 , || 1 1 2 

V — U = V + U . 



Finally, we have: 



|r-u|| 2 > — 1 1 v — u| I 2 = -llvll 2 + -Null 2 > -llrl 
i ii - 2 n ii 2 n ii 2 n n - 2 n i 



□ 
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Lecture 9 



Unique Games Hardness for MAXCUT 
Subhash Khot 



Scribe: Igor Gorodezky 
21 July, 2009 



9.1 Introduction: MAXCUT and Unique Games 

In this lecture we sketch the proof of one of the more remarkable consequences of the Unique Games 
Conjecture: MAX-CUT is inapproximable to any constant better than a, where 

a= min 2 W .87856 (9.1.1) 

-1<P<1 7T 1 — p 

is the approximation ratio of the Goemans- Williamson algorithm. 
9.1.1 The Goemans- Williamson algorithm 

Recall that given a graph G = (V, E, w) with edge weights Wij > 0, the MAX-CUT problem asks 
for S C V that maximizes X^es j<£S w n ( we can this the weight of the cut induced by S). We will 
write mc(G) for the maximum weight of a cut in G. 

MAX-CUT is NP-hard. The best known approximation algorithm for MAX-CUT is due to 
Goemans and Williamson [GW95] and is as follows: given G = (V, E, w), first solve the MAX-CUT 
SDP 

Wij j-^- (9.1.2) 

\\vi\\ 2 = 1, i = 1, . . . ,n 

Vi G R n , i = l,...,n 
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to get a set of unit vectors v\, . . . ,v n in M n . Then, uniformly sample a hyperplane through the 
origin and define S C V to be the set of i such that Vi lies "above" this hyperplane. 

We saw in Sanjeev's lecture (Lecture 1) that the Goemans- Williamson algorithm gives, in 
expectation, an (a — e)-approximation for any e > (this additive error of e stems from the fact 
that semidefinite programs must be solved to within some fixed, arbitrarily small accuracy). The 
algorithm can be derandomized (see [MH99]) to yield a deterministic (a — e)-approximation to 
MAX-CUT. 

A series of subsequent hardness results culminated in Hastad's PCP-based proof in [HasOl] that 
MAX-CUT is NP-hard to approximate to within 16/17 ~ .941. Then, roughly a decade after the 
publication of the Goemans- Williamson algorithm, Khot, Kindler, Mossel and O'Donnell proved in 
[KKMO07] that, assuming the Unique Games Conjecture, it is NP-hard to approximate MAX-CUT 
to within any factor greater than a. This suggests, as Khot et al. note, that the geometric nature 
of the Goemans- Williamson algorithm is intrinsic to the MAX-CUT problem. 

9.1.2 Label Cover and Unique Games 

The inapproximability of MAX-CUT is conditional on the Unique Games Conjecture, which we 
state in this section. 

A unique game £ is a bipartite graph with left-side vertex set V, right-side vertex set W, edge 
set E, and a set of labels of size M. Each edge (v,w) has an associated constraint function ir VjW 
which is a permutation of [M] (i.e. a bijection from the set of labels to itself). We will sometimes 
refer to a unique game with these parameters in the longhand C(V, W, E, [M], {ir v vl }). 

A labeling of a unique game £ is an assignment of a label from [M] to each vertex of £. A 
labeling satisfies the edge (v,w) if ir VtW maps the label of w to the label of v. We define 

opt(£) = max{ r | there exists a labeling of £ that satisfies r\E\ edges}. 

The unique Label Cover problem with parameter 5 is the problem of deciding, given a unique game 
£(V,W,E, [M],{Tr VtW }), whether opt(£) > 1 — 5 or opt(£) < 6. That is, given £, we are asked 
to decide whether there exists a labeling that satisfies nearly all edge constraints, or whether no 
labeling can satisfy more than a tiny fraction of them. We will write this decision problem as 
ULC((5). 

Intuition tells us that computing opt(£) should be a hard problem, but what about ULC(<5)? 
That is, if we are asked not to compute opt(£) but simply to decide whether it is very large or very 
small, does the problem become easier? The Unique Games Conjecture claims that the answer is no. 

The following is (a slightly weakened form of) the conjecture, first stated by Khot in [Kho02]. 

Unique Games Conjecture ([Kho02]) : For any 5 > there exists a constant M such that it 
is NP-hard to decide ULC(5) on instances with a label set of size M. 

9.1.3 The Main Result 

We are ready to present the main result of [KKMO07]. This result can be stated in the form of a 
PCP for the ULC(5) problem, by which we mean a probabilistically checkable proof system such 
that given a unique game £ and a proof, if opt(£) > 1 — 5 then the verifier accepts with high 
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V w 

Figure 9.1: A unique game with M = 3. The constraint tt is associated with the highlighted edge. 
The edge is satisfied by the labeling in the figure if vr(3) = 2. 

probability c (completeness), while if opt(£) < 5 then the verifier accepts with low probability s 
(soundness). As usual, the verifier only uses O(logn) random bits on an instance of size n. 

Theorem 9.1.1. For every p 6 (—1,0) and e > there exists 5 > such that there is a P CP for 
ULC(5) in which the verifier reads two bits from the proof and accepts iff they are unequal, and 
which has completeness 

. I-P 
C ^ — ~ e 

and soundness 

1 

s < — arccos(p) + e. 

Before sketching the proof of this theorem in Section 9.3, let us see how it implies the inapprox- 
imability of MAX-CUT. Recall that mc(G) is the maximum weight of a cut in G. 

Corollary 9.1.2. For every p 6 (—1,0) and e > there exists 5 > and a polynomial-time 
reduction from an instance C of ULC(5) to an instance G = (V,E,w) of MAX-CUT such that 

opt(£) > 1 - 5 =^ mc(G) > — ^ - e 
opt(£) < 5 => mc(G) < — arccos(p) + e. 

Proof. Given p and e, let 5 be the same as in Theorem 9.1.1. Given an instance C of ULC(<5), 
consider the PCP given by that theorem. Define the graph G to have the bits of the proof as 
vertices, 1 and create an edge between two bits if there is a non-zero probability of that pair of bits 
being sampled by the verifier. Finally, set w to be the trivial weight function that is 1 on all edges. 

Observe that a proof, which is an assignment of a value in {—1,1} to the bits, corresponds to 
a cut in G, and the number of edges crossing this cut is precisely the probability that this proof 
is accepted by the verifier. The claim now follows from the completeness and soundness of the 
PCP. □ 



1 As we will later see, if C is of the form C(V, W, E, [M], {k v ,w}) then there will be | W\2 M vertices in G. 
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Assuming the Unique Games Conjecture, for any 5 > there is some constant M such that it 
is NP-hard to decide ULC(<5) on instances with a label set of size M. Now, by standard arguments, 
Corollary 9.1.2 implies that it is NP-hard to approximate MAX-CUT to within 

arccos(/))/7r + e arccos (/?)/% 
(l-p)/2-e > (l-p)/2 

for any G (—1,0). Therefore, MAX-CUT is hard to approximate to within any constant larger 
than 

2 arccos(p) . 2 arccosf/?) 

mm = mm = a 

-l<p<0 7T 1 — p -1<p<1 vr 1 — p 

which is the promised inapproximability result. 

Let us turn our attention, then, to proving Theorem 9.1.1. The proof will rely on a highly 
nontrivial result in boolean Fourier analysis that we state in the next section. 



9.2 Majority is Stablest 

The proof of Theorem 9.1.1 makes crucial use of the Majority is Stablest (MIS) theorem, which is 
an extremal result in boolean Fourier analysis. In this section we state this theorem after defining 
the necessary concepts. 

We will use the common convention that bits take value in {—1, 1} rather than {0, 1} (in par- 
ticular, we identify x G {0,1} with y G { — 1,1} using the bijection y = (— l) x ). Thus, a boolean 
function is a map / : {—1, l} n —> { — 1, 1}- In what follows we will assume familiarity with the basic 
concepts of boolean Fourier analysis, as we lack the space for a thorough review of the subject; a 
reader seeking such a review is directed to the survey [O'D08]. 

We begin with several definitions. Given / : {—1, l} n —> { — 1, 1}, define the influence of x% on 
/ to be the probability over all n-bit strings that / changes value when the ith bit is flipped: 

Infi(/) = Prob [f(x)^ f(xi,...,Xi- 1 ,-x i ,x i+ i,...,x n )]. 
xe{-i,i} n 

It is not hard to show that 

Infi(/) = £ f(S) 2 (9.2.1) 

S\i£S 

and indeed, equation (9.2.1) can be used to define the influence of a variable on a non-boolean 
function / : {-1,1}" ^M. 

Given a bit string x and some p £ (—1,1), let us define a distribution over y G {— l,l} n by 
setting 

J Xi with probability 
1 — Xi with probability 

We write y ~ p x to mean a y sampled from such a distribution. Now, given a boolean function / 
and p G (—1,1), define the noise sensitivity of / at rate p to be 

NS P (/) = Prob [f(x) + f(y)] 
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where x is sampled uniformly and y ~ p x. It can be shown that 

NS p (/) = i-i£/(S)V S| . (9.2.2) 

s 

As before, equation (9.2.2) serves as the definition of noise sensitivity for non-boolean functions. 

We observe that if / is a dictator function, i.e. f(xi, . ■ ■ , x n ) = Xi for some i, then NS P (/) is 
exactly (1 — p)/2 since f(S) = when S ^ {x{\ and is 1 otherwise. If / is the majority function 
(the boolean function whose value on a bit string is equal to the value of the majority of the bits), 
then it can be shown (see [KKMO07] for references) that 

NS„(/) = -arccos(p) +o(l). 

Observe that when p € [0, 1), the noise sensitivity of a dictator function is lower than that of the 
majority function. The MIS theorem (proven in [MOO05]) tells us that if we disqualify dictators by 
restricting our attention to functions in which no coordinate has large influence, then the majority 
function achieves the smallest possible noise sensitivity. 

Theorem 9.2.1 (Majority is Stablest, [MOO05]). For every p G [0,1), e > there exists 5 such 
that if f : {-1, 1}™ -)• [-1, 1] with E [/] = and Inf;(/) < 5 Vi, then 

NSp(/) > — arccos p — e. 

Note the additional requirement that E [/] =0; such functions are called balanced. Our applica- 
tion requires the following corollary from [KKMO07] . It states that if we choose a negative rather 
than positive p, the majority function becomes the least stable. 

Corollary 9.2.2. For every p £ (— 1,0), e > there exists 6 such that if f : {— l,l} ra — > [—1,1] 
with Infi(/) < 5 Vi, then 

NSp(/) < — arccos p + e. 
Observe that / is no longer required to be balanced. 

Looking ahead to the proof of Theorem 9.1.1, the test that the verifier will perform on the PCP 
in the theorem will be, in a way, a noise-sensitivity test on a boolean function. Thus, we will be 
able to bound the soundness of this test by appealing to the MIS theorem. 

9.3 Proving Theorem 9.1.1 

In this section we sketch the construction of the PCP whose existence is claimed in Theorem 9.1.1. 
Recall that the PCP is for the problem ULC((5), so the verifier is given a unique game C(V, W, E, [M], {7r v , w }) 
and a proof that is accepted with high probability if opt(£) > 1 — 5 and accepted with low proba- 
bility if opt(£) < 5. The PCP will be parameterized by p £ (—1,0) and e > 0. 

It follows from the results of [KR08] that given C, we may assume with no loss of generality 
that all v E V have the same degree. Thus, uniformly sampling v £ V and then uniformly sampling 
a neighbor w £ W of v yields a uniformly random edge (v, w). Therefore, if we define a proof to be 
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a labeling of C that maximizes the proportion of satisfied edge constraints, and define the verifier's 
test to be uniformly sampling (v, w) and checking if this labeling satisfies ir v>w , then we would have 
a proof with completeness 1 — 5 and soundness 5. However, such a test involves sampling r2(log M) 
bits, and we require a test that samples only 2. 

We therefore look for a way to encode elements of the label set [M] in a way that will allow such 
a test. To this end, we will encode labels using the Long Code, which we first saw in Subhash's 
lecture (Lecture 7) on Hastad's 3-bit PCP. 

9.3.1 Motivation: the Long Code 

Recall that in the Long Code, the codeword encoding i G [M] is the truth-table for the dictator 
function f(x%, . . . , x n ) = X{. In our PCP, the proof will be a labeling of C with each label encoded 
using the Long Code. It remains to design a test for the verifier with the properties specified in 
Theorem 9.1.1. Before explicitly stating the test in the next section, we use this section to motivate 
its construction. 

Given a boolean function /, let us say that / is far from a dictator if all coordinates have 
negligible influence. It is not hard to design a 2-bit test that, given a truth-table for a function /, 
accepts with high probability if / is a dictator and with low probability if / is far from a dictator. 
The test required for our PCP must clearly do more than this, but for the moment let us warm up 
with this simpler problem. 

Consider the following noise-sensitivity test: sample x G { — 1, l} n uniformly, sample y ~ p x as 
in Section 9.2, and accept iff f(x) / /(y). By definition, the probability of accepting is NS P (/), 
the noise sensitivity of /. 

The completeness of this test is thus exactly the noise-sensitivity of a dictator function, which 
is (1 — p)/2. On the other hand, we can use Corollary 9.2.2 to bound the soundness: if / is far 
from a dictator its noise-sensitivity is at most arccos(/?)/-7r + e. 

Returning to our PCP, we require a 2-bit noise-sensitivity test with (almost) exactly these 
parameters that instead of testing whether a boolean function is a dictator or far from it, tests 
whether an encoded labeling of C (which consists of many boolean functions) satisfies many edge 
constraints, or far from it. 

9.3.2 The Test 

In this section we describe our PCP's verifier test. First, some notation. For x G {—1, 1} M and 
a bijection ir : [M] — > [M], let x o ir denote the string (ic^m, . . . , ^(m))- Given a unique game 
C(V, W, E, [M], {tt v ,w}) and the associated PCP proof (which the verifier expects is the Long Code 
encoding of a labeling), let f v be the Long Code encoding of the label given to v G V, and define 
f w for w G W analogously. 
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The 2-bit verifier test: 

1. Given C(V, W, E, [M], {ir VtW }) and a proof, sample v G V uniformly, then 
sample two of its neighbors w, w 1 G W uniformly and independently. 

2. Let 7r = ir VtW and n' = tt V)W i be the constraints for edges (v,w) and (v,w'). 

3. Sample x G {—1, 1} M uniformly and sample y ~ p x. 

4. Accept if /^(x o n) ^ /u/(y o 7r'). 



Completeness. Assume that opt(£) > 1 — <5 and that the proof given to the verifier encodes all 
labels correctly (i.e. as dictator functions). Let the labels of v,w,w' be i, j,f G [M], respectively. 
With probability at least 1 — 25, both (v, w) and (v, w') are satisfied by the labeling, which implies 
^(j) = ft'U') = *■ Conditioning on this event, we have 

f w (x o tt) = x^y) = Xi and /^/(y o vr') = y^^,) = y^. 

Since X{ = y, with probability (1 — p)/2, the test accepts with the same probability. We conclude 
that the completeness is at least (1 — 25) (1 — p)/2. Tweaking our choice of p, we conclude that 
completeness is at least (1 — p)/2 — e, as desired. 




Figure 9.2: The 2-bit verifier test samples v G V, two neighbors w,w' G W, then compares f w and 
/ m ' on certain inputs. 

Soundness. As usual, bounding the soundness is the difficult part; we only sketch the argu- 
ment. The proof is in the contrapositive direction: assuming that the test accepts with probability 
greater than arccos(/o) / it + e, we prove the existence of a labeling that satisfies many edges by 
exploiting the resulting Fourier-analytic properties of the boolean functions encoded in the proof 
(which is where Corollary 9.2.2 comes in). 
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The proof is as follows. Given v 6 V, let p v be the probability that the test accepts after 
choosing v from V. Then 



Pv = E 

w,w',x,y^ p x 

2 2«i,«i' 



1 - fw(xoir)f w >(yoir') 



E [/ tt (xo7r)/ m /(yo7r')l 

x,y^ p x L J 



Standard Fourier-analytic arguments can be used to show that 

J^nX 



from which it follows that 

Pv 



1-1e 

2 2ui,ui' 



1 51 



1 1 

2 ~ 2 

1 1 

2 ~ 2 



V E [/ U )(5)/ tu '(5') 



E 

Vj'r^V 



p \s\ 



where the last equality follows from the independence of w and w' (and w ~ v means that to is a 
neighbor of v). If we define a function g v : {—1, l} n — >■ [—1, 1] by 



then it is not hard to show that 



g v (z) = E [f w (z o tt v w )] 

w~v 

g v (S) = E \f w (S) ■ 



Therefore, returning to p v , we have 

1 1 



Pv 



2 2 

1 1 

2 ~ 2 



V E [/ w (5)l E [/US) 



,|5| 



= NS P (^) 

where the last equality is by equation (9.2.2). 

Recall that we assumed that the test is accepted with probability at least arccos(/3)/7r + e. 
Standard averaging arguments tell us that p v > arccos(/9)/-7r + e/2 for at least an e/2 fraction of 
v £ V. By the above, we have 

NS p (g v ) > arccos(yo)/7r + e/2 

for such v. Now we conclude by Corollary 9.2.2 (having tweaked e as necessary) that for such a 
v, g v has an influential coordinate. This fact can be used to show that for a constant fraction of 
neighbors w of v, f w has a small set of influential coordinates. These various influential coordinates 
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can be used to define a labeling that satisfies a large fraction of constraints; we direct the reader 
to [KKMO07] for the gory details. 

A final caveat: technically, if we want to prove that f w has a small set of influential coordinates 
for a constant fraction of neighbors w of v, it is not enough to assume that g v has a coordinate 
with large influence. What is required is for g v to have a coordinate with large low-degree influence, 
which is defined, in analogy to equation (9.2.1), as 

Rnf(/) = Y, f(Sf 
S\i£S,\S\<k 

for some constant k. If we define low-order noise sensitivity in obvious analogy to equation (9.2.2), 
it is possible to prove low-order analogues of the Majority is Stablest theorem and Corollary 9.2.2, 
which can then be used to formalize the argument that we have sketched. 

This completes the description of the PCP test and the proof of Theorem 9.1.1. 

9.4 The Big Picture 

The past few years have seen a flurry of powerful inapproximability results conditional on the 
Unique Games Conjecture. In [Rag08], Raghavendra exhibits a canonical semidefinite programming 
relaxation of an arbitrary CSP whose integrality gap, assuming UGC, is precisely equal to the best 
possible approximation ratio for that CSP. In FOCS 2009, Raghavendra and Steurer presented an 
efficient rounding scheme for these SDPs that achieves the integrality gap. 

UGC has been used to prove that Vertex Cover is conditionally inapproximable to within 2 — e 
(see [KR08]). This proof utilizes a theorem on the influence of boolean functions due to Friedgut. 
In addition, it was independetly shown by Khot and Vishnoi [KV05] and Chawla et al. [CKK + 06] 
that UGC implies the hardness of approximating Sparsest Cut to within any constant. These 
proofs, as expected, use theorems on the influence of boolean functions due to Kahn-Kalai-Linial 
and Bourgain. 

Underlying these results are surprising and fruitful connections between unique game reductions, 
semidefinite programming relaxations of CSPs, extremal problems in Fourier analysis, and isoperi- 
metric problems in geometry. The reader is directed to Section 5 of [KKMO07] for an insightful 
high-level discussion of how these connections are manifested in the particular case of MAX-CUT. 
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